2024-09-21T07:16:07.338 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T07:16:07.362 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:16:07.445 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} email: pdonnell@redhat.com first_in_suite: false job_id: '7913597' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: client snapdir: .customsnapfuse pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: snapdirname: .customsnapkernel syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaH59tidbgaJVyprt8oI92H5nfONl8lAebahhj3a6uhipeF4PAAM9J+5ETr9jsz//vOVOIVjNE831Rhyu9KMPA= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmDpRWpKgDUEh1EJap1QVIdk+KMgjerpQ0SDXdCjS0CrzbWOKN/vGIGIM5sKr9Xwmb0a+wFHveLuHgZQRu2bw4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T07:16:07.445 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T07:16:07.446 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T07:16:07.446 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T07:16:07.447 INFO:teuthology.task.internal:Checking packages... 2024-09-21T07:16:07.468 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T07:16:07.468 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T07:16:07.468 INFO:teuthology.packaging:ref: None 2024-09-21T07:16:07.468 INFO:teuthology.packaging:tag: None 2024-09-21T07:16:07.468 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:16:07.468 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:16:07.469 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:16:07.622 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T07:16:07.623 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T07:16:07.634 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T07:16:07.634 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T07:16:07.647 INFO:teuthology.task.internal:Saving configuration 2024-09-21T07:16:07.664 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T07:16:07.672 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T07:16:07.694 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:11:09.771836', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaH59tidbgaJVyprt8oI92H5nfONl8lAebahhj3a6uhipeF4PAAM9J+5ETr9jsz//vOVOIVjNE831Rhyu9KMPA='} 2024-09-21T07:16:07.718 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:11:09.773235', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmDpRWpKgDUEh1EJap1QVIdk+KMgjerpQ0SDXdCjS0CrzbWOKN/vGIGIM5sKr9Xwmb0a+wFHveLuHgZQRu2bw4='} 2024-09-21T07:16:07.718 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T07:16:07.729 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T07:16:07.729 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T07:16:07.729 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T07:16:07.794 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2b799f0160>, signals=[15]) 2024-09-21T07:16:07.794 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T07:16:07.804 INFO:teuthology.task.internal:Opening connections... 2024-09-21T07:16:07.805 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:16:07.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:16:07.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:16:07.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:16:07.959 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T07:16:07.970 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-09-21T07:16:07.989 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-09-21T07:16:07.989 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:16:08.047 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:16:08.048 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:16:08.048 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:16:08.048 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:16:08.048 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-09-21T07:16:08.076 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-09-21T07:16:08.094 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-09-21T07:16:08.095 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-09-21T07:16:08.155 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-09-21T07:16:08.155 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-09-21T07:16:08.155 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:16:08.156 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:16:08.157 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-09-21T07:16:08.175 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T07:16:08.189 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T07:16:08.251 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T07:16:08.252 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:16:08.254 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:16:08.271 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T07:16:08.280 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T07:16:08.281 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:16:08.312 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:16:08.406 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T07:16:08.461 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T07:16:08.476 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T07:16:08.477 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-21T07:16:08.477 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-21T07:16:08.477 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:16:08.477 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:16:08.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:16:08.494 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-21T07:16:08.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:16:08.495 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T07:16:08.552 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:16:08.553 INFO:teuthology.task.kernel:Running kernel on smithi105: 5.14.0-503.el9.x86_64 2024-09-21T07:16:08.553 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2024-09-21T07:16:08.554 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:16:08.554 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-503.el9.x86_64 2024-09-21T07:16:08.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-09-21T07:16:10.969 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T07:16:11.072 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.3 MB 00:00 2024-09-21T07:16:12.739 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-21T07:16:14.145 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 20 MB 00:02 2024-09-21T07:16:18.338 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-21T07:16:18.940 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-21T07:16:19.772 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 19 kB 00:00 2024-09-21T07:16:20.893 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T07:16:21.241 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-21T07:16:22.112 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T07:16:26.207 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-21T07:16:27.542 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-21T07:16:28.106 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T07:16:28.106 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T07:16:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T07:16:28.218 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T07:16:28.218 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-09-21T07:16:28.219 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T07:16:28.219 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T07:16:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T07:16:28.219 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T07:16:28.219 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-09-21T07:16:28.220 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2024-09-21T07:16:28.221 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:16:28.222 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 79 M 2024-09-21T07:16:28.222 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 126 M 2024-09-21T07:16:28.222 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T07:16:28.592 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 304 kB/s | 36 kB 00:00 2024-09-21T07:16:29.449 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T07:16:29.449 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T07:16:29.552 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T07:16:29.560 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:16:29.561 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-09-21T07:16:29.561 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2024-09-21T07:16:29.561 INFO:teuthology.orchestra.run.smithi105.stdout:Install 4 Packages 2024-09-21T07:16:29.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:16:29.561 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 79 M 2024-09-21T07:16:29.562 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 126 M 2024-09-21T07:16:29.562 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-09-21T07:16:29.977 INFO:teuthology.orchestra.run.smithi105.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 120 kB/s | 36 kB 00:00 2024-09-21T07:16:31.184 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.8 MB/s | 16 MB 00:02 2024-09-21T07:16:33.575 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.7 MB/s | 29 MB 00:04 2024-09-21T07:16:37.425 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-21T07:16:37.426 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:16:37.426 INFO:teuthology.orchestra.run.smithi112.stdout:Total 8.6 MB/s | 79 MB 00:09 2024-09-21T07:16:37.985 INFO:teuthology.orchestra.run.smithi105.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.9 MB/s | 16 MB 00:08 2024-09-21T07:16:38.110 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T07:16:38.479 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T07:16:38.479 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T07:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T07:16:39.443 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T07:16:41.873 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T07:16:42.803 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T07:16:42.885 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:16:45.655 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:16:45.797 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:16:50.683 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:16:50.731 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:16:55.325 INFO:teuthology.orchestra.run.smithi105.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 1.3 MB/s | 35 MB 00:25 2024-09-21T07:16:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 1.1 MB/s | 29 MB 00:25 2024-09-21T07:16:55.692 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:16:55.693 INFO:teuthology.orchestra.run.smithi105.stdout:Total 3.0 MB/s | 79 MB 00:26 2024-09-21T07:16:55.694 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:16:56.376 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-09-21T07:16:56.741 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-09-21T07:16:56.741 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-09-21T07:16:57.692 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-09-21T07:16:57.692 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-09-21T07:17:00.118 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-09-21T07:17:00.920 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T07:17:01.018 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:17:03.776 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:17:03.913 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:17:08.743 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:17:08.788 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:13.642 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:33.720 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:33.760 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:34.153 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T07:17:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:17:34.154 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:17:34.995 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:17:34.996 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T07:17:35.344 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T07:17:35.482 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:35.482 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T07:17:35.482 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T07:17:35.482 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T07:17:35.483 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T07:17:35.483 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-21T07:17:35.483 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T07:17:35.483 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T07:17:35.541 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:35.541 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T07:17:35.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2024-09-21T07:17:35.581 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2024-09-21T07:17:35.581 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 82.el9 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8274670 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2024-09-21T07:17:35.582 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:17:35.583 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:17:35.584 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:17:35.584 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-508.el9.x86_64 2024-09-21T07:17:35.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:17:36.248 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-09-21T07:17:37.595 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:17:37.615 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-09-21T07:17:37.617 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T07:17:37.633 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.30cW2sHyJU 2024-09-21T07:17:37.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.30cW2sHyJU 2024-09-21T07:17:37.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.30cW2sHyJU 2024-09-21T07:17:37.847 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.30cW2sHyJU is 6KB 2024-09-21T07:17:37.894 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.30cW2sHyJU 2024-09-21T07:17:37.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:17:37.971 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:17:37.971 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:17:37.971 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:17:37.971 INFO:teuthology.orchestra.run.smithi112.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T07:17:37.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T07:17:38.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-09-21T07:17:51.840 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:51.880 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:52.229 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:52.229 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T07:17:52.229 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T07:17:52.229 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T07:17:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T07:17:53.032 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:17:53.032 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:17:53.033 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-09-21T07:17:53.395 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T07:17:53.486 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.486 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T07:17:53.486 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T07:17:53.486 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T07:17:53.486 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T07:17:53.486 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T07:17:53.486 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T07:17:53.486 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T07:17:53.544 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T07:17:53.544 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T07:17:53.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.06 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 82.el9 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2024-09-21T07:17:53.576 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Group : Unspecified 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 8274670 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : builder@centos.org 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:17:53.577 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:17:53.578 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:17:53.579 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:17:53.579 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 5.14.0-508.el9.x86_64 2024-09-21T07:17:53.579 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:17:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-09-21T07:17:55.581 INFO:teuthology.orchestra.run.smithi105.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:17:55.613 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-09-21T07:17:55.615 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-21T07:17:55.631 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.EEtkUSIA8G 2024-09-21T07:17:55.631 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EEtkUSIA8G 2024-09-21T07:17:55.698 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.EEtkUSIA8G 2024-09-21T07:17:55.838 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.EEtkUSIA8G is 6KB 2024-09-21T07:17:55.886 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.EEtkUSIA8G 2024-09-21T07:17:55.901 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:17:55.964 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:17:55.964 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:17:55.964 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:17:55.964 INFO:teuthology.orchestra.run.smithi105.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T07:17:55.966 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T07:17:56.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-09-21T07:18:08.130 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T07:18:08.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:18:08.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:18:26.151 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T07:18:26.152 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:18:26.152 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:18:26.543 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-09-21T07:18:35.550 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:18:35.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:18:44.719 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-21T07:18:53.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:18:53.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:18:56.783 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-21T07:19:08.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:19:08.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:19:27.215 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2024-09-21T07:19:35.561 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T07:19:42.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:19:42.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:19:42.507 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-21T07:19:42.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:19:42.949 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T07:19:42.949 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-09-21T07:19:42.965 INFO:teuthology.orchestra.run.smithi105.stdout:5.14.0-508.el9.x86_64 2024-09-21T07:19:42.965 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T07:19:42.965 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:19:42.965 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:19:43.967 DEBUG:teuthology.parallel:result is None 2024-09-21T07:19:47.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:19:47.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:19:47.994 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T07:19:48.377 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:19:48.377 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-508.el9.x86_64"... 2024-09-21T07:19:48.378 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-09-21T07:19:48.396 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-508.el9.x86_64 2024-09-21T07:19:48.396 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T07:19:48.396 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:19:48.396 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:19:49.397 DEBUG:teuthology.parallel:result is None 2024-09-21T07:19:49.397 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T07:19:49.408 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T07:19:49.408 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:19:49.411 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:19:49.429 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T07:19:49.446 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T07:19:49.474 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T07:19:49.475 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:19:49.477 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:19:49.531 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T07:19:49.540 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T07:19:49.540 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:19:49.550 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:19:49.605 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:19:49.631 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:19:49.634 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:19:49.653 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:19:49.655 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T07:19:49.661 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T07:19:49.662 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:19:49.664 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:19:49.691 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T07:19:49.700 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T07:19:49.701 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:19:49.734 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:19:49.751 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:19:49.836 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:19:49.891 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:19:49.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:19:50.016 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:19:50.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:19:50.082 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:19:50.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:19:50.179 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:19:50.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:19:50.305 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:19:50.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:19:50.371 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-09-21T07:19:50.373 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-09-21T07:19:50.414 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:19:50.466 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:19:50.935 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T07:19:50.944 INFO:teuthology.task.internal:Starting timer... 2024-09-21T07:19:50.944 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T07:19:50.953 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T07:19:50.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-09-21T07:19:51.029 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-09-21T07:19:51.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-09-21T07:19:51.097 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-09-21T07:19:51.099 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T07:19:51.099 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-09-21T07:19:51.135 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-09-21T07:19:51.135 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-09-21T07:19:51.153 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-09-21T07:19:51.154 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2024-09-21T07:19:51.154 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:19:51.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:19:51.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:19:51.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:19:51.233 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T07:19:51.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:19:51.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:19:51.315 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T07:19:51.324 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T07:19:51.335 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T07:19:51.335 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T07:23:33.875 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2024-09-21T07:23:33.877 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:23:33.879 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:23:33.961 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-09-21T07:23:34.046 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-09-21T07:23:34.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:23:34.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:23:34.121 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-09-21T07:23:34.207 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-09-21T07:23:34.207 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T07:23:34.217 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T07:23:34.217 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:23:34.217 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:23:34.220 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:23:34.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:23:34.248 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:23:34.263 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:23:34.278 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:23:34.287 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-09-21T07:23:34.293 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:23:34.298 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-09-21T07:23:34.313 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:23:34.318 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-09-21T07:23:34.327 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:23:34.329 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-09-21T07:23:34.343 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:23:34.357 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:23:34.384 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:23:34.387 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:23:34.387 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-09-21T07:23:34.387 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.388 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.388 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.388 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.412 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:23:34.417 INFO:teuthology.run_tasks:Running task install... 2024-09-21T07:23:34.431 DEBUG:teuthology.task.install:project ceph 2024-09-21T07:23:34.431 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T07:23:34.431 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T07:23:34.431 INFO:teuthology.task.install:Using flavor: default 2024-09-21T07:23:34.436 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T07:23:34.436 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T07:23:34.437 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:23:34.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:23:34.439 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:23:34.439 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:23:34.586 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:23:34.586 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:23:34.592 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:23:34.592 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:23:34.700 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:23:34.700 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:23:34.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:23:34.703 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:23:34.703 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:23:34.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:23:34.726 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:23:34.727 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:23:34.730 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:23:34.730 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:23:34.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:23:34.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:23:34.867 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:23:34.869 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:23:34.931 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-09-21T07:23:34.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-09-21T07:23:34.933 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-09-21T07:23:34.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-09-21T07:23:35.156 INFO:teuthology.orchestra.run.smithi112.stdout:45 files removed 2024-09-21T07:23:35.162 INFO:teuthology.orchestra.run.smithi105.stdout:45 files removed 2024-09-21T07:23:35.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:23:35.188 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:23:35.737 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-21T07:23:35.744 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-21T07:23:36.029 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T07:23:36.045 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T07:23:36.305 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T07:23:36.329 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T07:23:37.014 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T07:23:39.296 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.3 MB 00:02 2024-09-21T07:23:39.915 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-21T07:23:41.022 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-21T07:23:45.271 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-21T07:23:45.548 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-21T07:23:46.725 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 19 kB 00:00 2024-09-21T07:23:47.099 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-21T07:23:47.729 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-21T07:23:47.920 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-21T07:23:53.194 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T07:23:53.401 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-21T07:23:55.417 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.417 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.431 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:23:55.432 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.432 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:23:55.433 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.435 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:23:55.438 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.439 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.504 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-09-21T07:23:55.537 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T07:23:55.537 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:23:55.538 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:23:55.539 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:23:55.540 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:23:55.541 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:23:55.542 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:23:55.543 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:23:55.544 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:23:55.545 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:23:55.546 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:23:55.547 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:23:55.548 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:23:55.549 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:23:55.550 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:23:55.551 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:23:55.552 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:23:55.553 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:23:55.554 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:23:55.555 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:23:55.556 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:23:55.557 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:Install 149 Packages 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 8 Packages 2024-09-21T07:23:55.558 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:23:55.559 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 228 M 2024-09-21T07:23:55.559 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-09-21T07:23:55.621 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.622 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.635 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:23:55.636 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:23:55.637 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:23:55.638 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.640 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:23:55.643 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.644 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:23:55.708 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-09-21T07:23:55.728 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:23:55.729 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:23:55.730 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:23:55.731 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:23:55.732 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:23:55.733 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:23:55.734 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:23:55.735 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:23:55.736 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:23:55.737 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:23:55.738 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:23:55.739 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout:Install 149 Packages 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 8 Packages 2024-09-21T07:23:55.740 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:23:55.741 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 228 M 2024-09-21T07:23:55.741 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-09-21T07:23:56.080 INFO:teuthology.orchestra.run.smithi112.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 53 kB/s | 6.6 kB 00:00 2024-09-21T07:23:56.194 INFO:teuthology.orchestra.run.smithi105.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 51 kB/s | 6.6 kB 00:00 2024-09-21T07:23:56.339 INFO:teuthology.orchestra.run.smithi112.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.3 MB/s | 873 kB 00:00 2024-09-21T07:23:56.447 INFO:teuthology.orchestra.run.smithi112.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T07:23:56.472 INFO:teuthology.orchestra.run.smithi112.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.1 MB/s | 148 kB 00:00 2024-09-21T07:23:56.606 INFO:teuthology.orchestra.run.smithi112.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 14 MB/s | 2.3 MB 00:00 2024-09-21T07:23:56.635 INFO:teuthology.orchestra.run.smithi105.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.9 MB/s | 873 kB 00:00 2024-09-21T07:23:56.757 INFO:teuthology.orchestra.run.smithi112.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-09-21T07:23:56.811 INFO:teuthology.orchestra.run.smithi105.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 847 kB/s | 148 kB 00:00 2024-09-21T07:23:57.049 INFO:teuthology.orchestra.run.smithi112.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T07:23:57.124 INFO:teuthology.orchestra.run.smithi112.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 9.6 MB/s | 5.0 MB 00:00 2024-09-21T07:23:57.170 INFO:teuthology.orchestra.run.smithi105.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 6.4 MB/s | 2.3 MB 00:00 2024-09-21T07:23:57.199 INFO:teuthology.orchestra.run.smithi112.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 336 kB/s | 25 kB 00:00 2024-09-21T07:23:57.403 INFO:teuthology.orchestra.run.smithi105.stdout:(5/157): ceph-base-19.3.0-5120.g18432218.el9.x8 4.1 MB/s | 5.5 MB 00:01 2024-09-21T07:23:57.532 INFO:teuthology.orchestra.run.smithi112.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 22 MB/s | 10 MB 00:00 2024-09-21T07:23:57.537 INFO:teuthology.orchestra.run.smithi105.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.1 MB/s | 1.5 MB 00:00 2024-09-21T07:23:57.599 INFO:teuthology.orchestra.run.smithi112.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T07:23:57.675 INFO:teuthology.orchestra.run.smithi112.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 361 kB/s | 27 kB 00:00 2024-09-21T07:23:57.750 INFO:teuthology.orchestra.run.smithi112.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T07:23:57.859 INFO:teuthology.orchestra.run.smithi112.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T07:23:57.942 INFO:teuthology.orchestra.run.smithi112.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T07:23:58.026 INFO:teuthology.orchestra.run.smithi112.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T07:23:58.260 INFO:teuthology.orchestra.run.smithi112.stdout:(17/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 12 MB/s | 18 MB 00:01 2024-09-21T07:23:58.285 INFO:teuthology.orchestra.run.smithi112.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 1.9 MB/s | 507 kB 00:00 2024-09-21T07:23:58.352 INFO:teuthology.orchestra.run.smithi112.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 676 kB/s | 45 kB 00:00 2024-09-21T07:23:58.435 INFO:teuthology.orchestra.run.smithi112.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T07:23:58.454 INFO:teuthology.orchestra.run.smithi105.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 8.3 MB/s | 20 MB 00:02 2024-09-21T07:23:58.552 INFO:teuthology.orchestra.run.smithi112.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.3 MB 00:00 2024-09-21T07:23:58.577 INFO:teuthology.orchestra.run.smithi112.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.1 MB/s | 161 kB 00:00 2024-09-21T07:23:58.636 INFO:teuthology.orchestra.run.smithi112.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.8 MB/s | 321 kB 00:00 2024-09-21T07:23:58.662 INFO:teuthology.orchestra.run.smithi105.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 3.9 MB/s | 5.0 MB 00:01 2024-09-21T07:23:58.669 INFO:teuthology.orchestra.run.smithi112.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 3.3 MB/s | 307 kB 00:00 2024-09-21T07:23:58.703 INFO:teuthology.orchestra.run.smithi112.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T07:23:58.729 INFO:teuthology.orchestra.run.smithi105.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 378 kB/s | 25 kB 00:00 2024-09-21T07:23:58.753 INFO:teuthology.orchestra.run.smithi112.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.0 MB/s | 86 kB 00:00 2024-09-21T07:23:58.979 INFO:teuthology.orchestra.run.smithi112.stdout:(27/157): rbd-mirror-19.3.0-5120.g18432218.el9. 11 MB/s | 3.1 MB 00:00 2024-09-21T07:23:59.037 INFO:teuthology.orchestra.run.smithi112.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 587 kB/s | 34 kB 00:00 2024-09-21T07:23:59.105 INFO:teuthology.orchestra.run.smithi105.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 16 MB/s | 10 MB 00:00 2024-09-21T07:23:59.121 INFO:teuthology.orchestra.run.smithi112.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T07:23:59.155 INFO:teuthology.orchestra.run.smithi105.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T07:23:59.230 INFO:teuthology.orchestra.run.smithi112.stdout:(30/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 357 kB/s | 170 kB 00:00 2024-09-21T07:23:59.322 INFO:teuthology.orchestra.run.smithi105.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 10 MB/s | 18 MB 00:01 2024-09-21T07:23:59.347 INFO:teuthology.orchestra.run.smithi105.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 141 kB/s | 27 kB 00:00 2024-09-21T07:23:59.405 INFO:teuthology.orchestra.run.smithi105.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 382 kB/s | 32 kB 00:00 2024-09-21T07:23:59.456 INFO:teuthology.orchestra.run.smithi105.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 6.9 MB/s | 766 kB 00:00 2024-09-21T07:23:59.498 INFO:teuthology.orchestra.run.smithi105.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T07:23:59.540 INFO:teuthology.orchestra.run.smithi105.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.5 MB/s | 127 kB 00:00 2024-09-21T07:23:59.590 INFO:teuthology.orchestra.run.smithi105.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 5.4 MB/s | 507 kB 00:00 2024-09-21T07:23:59.674 INFO:teuthology.orchestra.run.smithi105.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 543 kB/s | 45 kB 00:00 2024-09-21T07:23:59.747 INFO:teuthology.orchestra.run.smithi112.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T07:23:59.749 INFO:teuthology.orchestra.run.smithi105.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T07:23:59.797 INFO:teuthology.orchestra.run.smithi112.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T07:23:59.816 INFO:teuthology.orchestra.run.smithi105.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.4 MB/s | 161 kB 00:00 2024-09-21T07:23:59.875 INFO:teuthology.orchestra.run.smithi105.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T07:23:59.941 INFO:teuthology.orchestra.run.smithi112.stdout:(33/157): ceph-mgr-dashboard-19.3.0-5120.g18432 17 MB/s | 14 MB 00:00 2024-09-21T07:23:59.942 INFO:teuthology.orchestra.run.smithi105.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.3 MB 00:00 2024-09-21T07:23:59.964 INFO:teuthology.orchestra.run.smithi112.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 295 kB/s | 49 kB 00:00 2024-09-21T07:23:59.975 INFO:teuthology.orchestra.run.smithi105.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 3.0 MB/s | 307 kB 00:00 2024-09-21T07:23:59.989 INFO:teuthology.orchestra.run.smithi112.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 328 kB/s | 16 kB 00:00 2024-09-21T07:24:00.000 INFO:teuthology.orchestra.run.smithi105.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T07:24:00.024 INFO:teuthology.orchestra.run.smithi112.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T07:24:00.026 INFO:teuthology.orchestra.run.smithi105.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T07:24:00.065 INFO:teuthology.orchestra.run.smithi112.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 10 MB/s | 784 kB 00:00 2024-09-21T07:24:00.076 INFO:teuthology.orchestra.run.smithi105.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T07:24:00.090 INFO:teuthology.orchestra.run.smithi112.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 275 kB/s | 18 kB 00:00 2024-09-21T07:24:00.126 INFO:teuthology.orchestra.run.smithi105.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 686 kB/s | 34 kB 00:00 2024-09-21T07:24:00.185 INFO:teuthology.orchestra.run.smithi105.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 17 MB/s | 3.1 MB 00:00 2024-09-21T07:24:00.210 INFO:teuthology.orchestra.run.smithi105.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.8 MB/s | 154 kB 00:00 2024-09-21T07:24:00.441 INFO:teuthology.orchestra.run.smithi112.stdout:(39/157): ceph-test-19.3.0-5120.g18432218.el9.x 12 MB/s | 40 MB 00:03 2024-09-21T07:24:00.574 INFO:teuthology.orchestra.run.smithi112.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 157 kB/s | 80 kB 00:00 2024-09-21T07:24:00.599 INFO:teuthology.orchestra.run.smithi112.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-21T07:24:00.619 INFO:teuthology.orchestra.run.smithi105.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 18 MB/s | 7.4 MB 00:00 2024-09-21T07:24:00.633 INFO:teuthology.orchestra.run.smithi112.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 801 kB 00:00 2024-09-21T07:24:00.675 INFO:teuthology.orchestra.run.smithi112.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T07:24:00.678 INFO:teuthology.orchestra.run.smithi105.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.9 MB/s | 292 kB 00:00 2024-09-21T07:24:00.700 INFO:teuthology.orchestra.run.smithi112.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T07:24:00.728 INFO:teuthology.orchestra.run.smithi105.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 989 kB/s | 49 kB 00:00 2024-09-21T07:24:00.742 INFO:teuthology.orchestra.run.smithi112.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-21T07:24:00.767 INFO:teuthology.orchestra.run.smithi112.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-21T07:24:00.778 INFO:teuthology.orchestra.run.smithi105.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T07:24:00.784 INFO:teuthology.orchestra.run.smithi112.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T07:24:00.809 INFO:teuthology.orchestra.run.smithi112.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-21T07:24:00.929 INFO:teuthology.orchestra.run.smithi105.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 19 MB/s | 14 MB 00:00 2024-09-21T07:24:00.954 INFO:teuthology.orchestra.run.smithi105.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.6 MB/s | 285 kB 00:00 2024-09-21T07:24:00.968 INFO:teuthology.orchestra.run.smithi112.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 676 kB/s | 107 kB 00:00 2024-09-21T07:24:01.013 INFO:teuthology.orchestra.run.smithi105.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.2 MB/s | 784 kB 00:00 2024-09-21T07:24:01.018 INFO:teuthology.orchestra.run.smithi112.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-21T07:24:01.058 INFO:teuthology.orchestra.run.smithi105.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 176 kB/s | 18 kB 00:00 2024-09-21T07:24:01.060 INFO:teuthology.orchestra.run.smithi112.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-21T07:24:01.127 INFO:teuthology.orchestra.run.smithi112.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-21T07:24:01.152 INFO:teuthology.orchestra.run.smithi112.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-21T07:24:01.163 INFO:teuthology.orchestra.run.smithi105.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-21T07:24:01.177 INFO:teuthology.orchestra.run.smithi112.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 1.9 MB/s | 313 kB 00:00 2024-09-21T07:24:01.203 INFO:teuthology.orchestra.run.smithi112.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-21T07:24:01.222 INFO:teuthology.orchestra.run.smithi105.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T07:24:01.228 INFO:teuthology.orchestra.run.smithi112.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T07:24:01.255 INFO:teuthology.orchestra.run.smithi105.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.0 MB/s | 801 kB 00:00 2024-09-21T07:24:01.287 INFO:teuthology.orchestra.run.smithi112.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T07:24:01.289 INFO:teuthology.orchestra.run.smithi105.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-21T07:24:01.312 INFO:teuthology.orchestra.run.smithi112.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T07:24:01.331 INFO:teuthology.orchestra.run.smithi105.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-21T07:24:01.337 INFO:teuthology.orchestra.run.smithi112.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-09-21T07:24:01.398 INFO:teuthology.orchestra.run.smithi105.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-21T07:24:01.448 INFO:teuthology.orchestra.run.smithi105.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T07:24:01.479 INFO:teuthology.orchestra.run.smithi112.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-21T07:24:01.513 INFO:teuthology.orchestra.run.smithi112.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-21T07:24:01.540 INFO:teuthology.orchestra.run.smithi105.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-21T07:24:01.563 INFO:teuthology.orchestra.run.smithi112.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-09-21T07:24:01.599 INFO:teuthology.orchestra.run.smithi105.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-09-21T07:24:01.605 INFO:teuthology.orchestra.run.smithi112.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 195 kB/s | 24 kB 00:00 2024-09-21T07:24:01.688 INFO:teuthology.orchestra.run.smithi112.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T07:24:01.714 INFO:teuthology.orchestra.run.smithi112.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-21T07:24:01.731 INFO:teuthology.orchestra.run.smithi112.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 921 kB/s | 38 kB 00:00 2024-09-21T07:24:01.783 INFO:teuthology.orchestra.run.smithi105.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-09-21T07:24:01.825 INFO:teuthology.orchestra.run.smithi105.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-21T07:24:01.850 INFO:teuthology.orchestra.run.smithi105.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-21T07:24:01.875 INFO:teuthology.orchestra.run.smithi105.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T07:24:01.881 INFO:teuthology.orchestra.run.smithi112.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-09-21T07:24:01.909 INFO:teuthology.orchestra.run.smithi105.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-21T07:24:01.934 INFO:teuthology.orchestra.run.smithi105.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-21T07:24:01.976 INFO:teuthology.orchestra.run.smithi105.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-09-21T07:24:02.018 INFO:teuthology.orchestra.run.smithi105.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-21T07:24:02.077 INFO:teuthology.orchestra.run.smithi105.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-21T07:24:02.102 INFO:teuthology.orchestra.run.smithi105.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-21T07:24:02.127 INFO:teuthology.orchestra.run.smithi105.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-21T07:24:02.153 INFO:teuthology.orchestra.run.smithi105.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-09-21T07:24:02.187 INFO:teuthology.orchestra.run.smithi105.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-21T07:24:02.212 INFO:teuthology.orchestra.run.smithi105.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-21T07:24:02.223 INFO:teuthology.orchestra.run.smithi112.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-21T07:24:02.237 INFO:teuthology.orchestra.run.smithi105.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-21T07:24:02.262 INFO:teuthology.orchestra.run.smithi105.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-21T07:24:02.288 INFO:teuthology.orchestra.run.smithi105.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T07:24:02.322 INFO:teuthology.orchestra.run.smithi105.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T07:24:02.347 INFO:teuthology.orchestra.run.smithi105.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-21T07:24:02.457 INFO:teuthology.orchestra.run.smithi112.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.8 MB/s | 910 kB 00:00 2024-09-21T07:24:02.464 INFO:teuthology.orchestra.run.smithi105.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T07:24:02.531 INFO:teuthology.orchestra.run.smithi105.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 24 MB/s | 4.9 MB 00:00 2024-09-21T07:24:02.557 INFO:teuthology.orchestra.run.smithi112.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-21T07:24:02.573 INFO:teuthology.orchestra.run.smithi105.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T07:24:02.615 INFO:teuthology.orchestra.run.smithi105.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-21T07:24:02.640 INFO:teuthology.orchestra.run.smithi105.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T07:24:02.699 INFO:teuthology.orchestra.run.smithi105.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-21T07:24:02.749 INFO:teuthology.orchestra.run.smithi112.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-09-21T07:24:02.774 INFO:teuthology.orchestra.run.smithi105.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-21T07:24:02.799 INFO:teuthology.orchestra.run.smithi105.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-09-21T07:24:02.841 INFO:teuthology.orchestra.run.smithi105.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T07:24:02.925 INFO:teuthology.orchestra.run.smithi105.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-21T07:24:02.950 INFO:teuthology.orchestra.run.smithi105.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T07:24:02.974 INFO:teuthology.orchestra.run.smithi112.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.9 MB/s | 4.9 MB 00:01 2024-09-21T07:24:02.975 INFO:teuthology.orchestra.run.smithi105.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T07:24:03.001 INFO:teuthology.orchestra.run.smithi105.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-21T07:24:03.007 INFO:teuthology.orchestra.run.smithi112.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-21T07:24:03.026 INFO:teuthology.orchestra.run.smithi105.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T07:24:03.032 INFO:teuthology.orchestra.run.smithi112.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T07:24:03.051 INFO:teuthology.orchestra.run.smithi105.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T07:24:03.077 INFO:teuthology.orchestra.run.smithi105.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-21T07:24:03.083 INFO:teuthology.orchestra.run.smithi112.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T07:24:03.127 INFO:teuthology.orchestra.run.smithi105.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-21T07:24:03.191 INFO:teuthology.orchestra.run.smithi112.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T07:24:03.202 INFO:teuthology.orchestra.run.smithi105.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-21T07:24:03.228 INFO:teuthology.orchestra.run.smithi105.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-21T07:24:03.262 INFO:teuthology.orchestra.run.smithi105.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T07:24:03.287 INFO:teuthology.orchestra.run.smithi105.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-09-21T07:24:03.300 INFO:teuthology.orchestra.run.smithi112.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-21T07:24:03.320 INFO:teuthology.orchestra.run.smithi105.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T07:24:03.346 INFO:teuthology.orchestra.run.smithi105.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T07:24:03.350 INFO:teuthology.orchestra.run.smithi112.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T07:24:03.371 INFO:teuthology.orchestra.run.smithi105.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T07:24:03.396 INFO:teuthology.orchestra.run.smithi105.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T07:24:03.400 INFO:teuthology.orchestra.run.smithi112.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-21T07:24:03.421 INFO:teuthology.orchestra.run.smithi105.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T07:24:03.464 INFO:teuthology.orchestra.run.smithi105.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T07:24:03.484 INFO:teuthology.orchestra.run.smithi112.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-21T07:24:03.505 INFO:teuthology.orchestra.run.smithi105.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T07:24:03.534 INFO:teuthology.orchestra.run.smithi112.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-21T07:24:03.564 INFO:teuthology.orchestra.run.smithi105.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T07:24:03.615 INFO:teuthology.orchestra.run.smithi105.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 724 kB/s | 36 kB 00:00 2024-09-21T07:24:03.773 INFO:teuthology.orchestra.run.smithi105.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-21T07:24:03.793 INFO:teuthology.orchestra.run.smithi112.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-21T07:24:03.798 INFO:teuthology.orchestra.run.smithi105.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-21T07:24:03.824 INFO:teuthology.orchestra.run.smithi105.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-09-21T07:24:03.849 INFO:teuthology.orchestra.run.smithi105.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T07:24:03.874 INFO:teuthology.orchestra.run.smithi105.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T07:24:03.885 INFO:teuthology.orchestra.run.smithi112.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-21T07:24:03.908 INFO:teuthology.orchestra.run.smithi105.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T07:24:03.933 INFO:teuthology.orchestra.run.smithi105.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-09-21T07:24:03.969 INFO:teuthology.orchestra.run.smithi112.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T07:24:04.009 INFO:teuthology.orchestra.run.smithi105.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-21T07:24:04.010 INFO:teuthology.orchestra.run.smithi112.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-09-21T07:24:04.067 INFO:teuthology.orchestra.run.smithi105.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-21T07:24:04.101 INFO:teuthology.orchestra.run.smithi105.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T07:24:04.103 INFO:teuthology.orchestra.run.smithi112.stdout:(86/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-09-21T07:24:04.126 INFO:teuthology.orchestra.run.smithi105.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-21T07:24:04.128 INFO:teuthology.orchestra.run.smithi112.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-09-21T07:24:04.186 INFO:teuthology.orchestra.run.smithi112.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T07:24:04.212 INFO:teuthology.orchestra.run.smithi112.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-21T07:24:04.235 INFO:teuthology.orchestra.run.smithi105.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T07:24:04.260 INFO:teuthology.orchestra.run.smithi105.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T07:24:04.262 INFO:teuthology.orchestra.run.smithi112.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T07:24:04.287 INFO:teuthology.orchestra.run.smithi112.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-21T07:24:04.293 INFO:teuthology.orchestra.run.smithi105.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-21T07:24:04.312 INFO:teuthology.orchestra.run.smithi112.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T07:24:04.319 INFO:teuthology.orchestra.run.smithi105.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-09-21T07:24:04.353 INFO:teuthology.orchestra.run.smithi105.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2024-09-21T07:24:04.363 INFO:teuthology.orchestra.run.smithi112.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-21T07:24:04.386 INFO:teuthology.orchestra.run.smithi105.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 728 kB/s | 67 kB 00:00 2024-09-21T07:24:04.413 INFO:teuthology.orchestra.run.smithi112.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-21T07:24:04.428 INFO:teuthology.orchestra.run.smithi105.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T07:24:04.454 INFO:teuthology.orchestra.run.smithi105.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:24:04.472 INFO:teuthology.orchestra.run.smithi112.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T07:24:04.546 INFO:teuthology.orchestra.run.smithi112.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-09-21T07:24:04.546 INFO:teuthology.orchestra.run.smithi105.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T07:24:04.546 INFO:teuthology.orchestra.run.smithi105.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T07:24:04.546 INFO:teuthology.orchestra.run.smithi105.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-21T07:24:04.563 INFO:teuthology.orchestra.run.smithi105.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T07:24:04.572 INFO:teuthology.orchestra.run.smithi112.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-21T07:24:04.597 INFO:teuthology.orchestra.run.smithi105.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-21T07:24:04.622 INFO:teuthology.orchestra.run.smithi105.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T07:24:04.647 INFO:teuthology.orchestra.run.smithi105.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T07:24:04.656 INFO:teuthology.orchestra.run.smithi112.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-21T07:24:04.672 INFO:teuthology.orchestra.run.smithi105.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T07:24:04.698 INFO:teuthology.orchestra.run.smithi105.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-21T07:24:04.706 INFO:teuthology.orchestra.run.smithi112.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T07:24:04.756 INFO:teuthology.orchestra.run.smithi112.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T07:24:04.815 INFO:teuthology.orchestra.run.smithi112.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T07:24:04.874 INFO:teuthology.orchestra.run.smithi112.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-09-21T07:24:05.057 INFO:teuthology.orchestra.run.smithi105.stdout:(125/157): ceph-test-19.3.0-5120.g18432218.el9. 6.3 MB/s | 40 MB 00:06 2024-09-21T07:24:05.082 INFO:teuthology.orchestra.run.smithi105.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 46 kB/s | 18 kB 00:00 2024-09-21T07:24:05.115 INFO:teuthology.orchestra.run.smithi105.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 755 kB/s | 19 kB 00:00 2024-09-21T07:24:05.157 INFO:teuthology.orchestra.run.smithi112.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-21T07:24:05.160 INFO:teuthology.orchestra.run.smithi105.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 459 kB/s | 20 kB 00:00 2024-09-21T07:24:05.191 INFO:teuthology.orchestra.run.smithi105.stdout:(129/157): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-09-21T07:24:05.225 INFO:teuthology.orchestra.run.smithi105.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T07:24:05.258 INFO:teuthology.orchestra.run.smithi105.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T07:24:05.284 INFO:teuthology.orchestra.run.smithi105.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T07:24:05.308 INFO:teuthology.orchestra.run.smithi112.stdout:(104/157): python3-numpy-1.20.1-5.el9.x86_64.rp 2.9 MB/s | 5.1 MB 00:01 2024-09-21T07:24:05.317 INFO:teuthology.orchestra.run.smithi105.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2024-09-21T07:24:05.343 INFO:teuthology.orchestra.run.smithi105.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-21T07:24:05.358 INFO:teuthology.orchestra.run.smithi112.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-21T07:24:05.376 INFO:teuthology.orchestra.run.smithi105.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-21T07:24:05.383 INFO:teuthology.orchestra.run.smithi112.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-21T07:24:05.400 INFO:teuthology.orchestra.run.smithi112.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-21T07:24:05.401 INFO:teuthology.orchestra.run.smithi105.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T07:24:05.451 INFO:teuthology.orchestra.run.smithi105.stdout:(137/157): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-09-21T07:24:05.477 INFO:teuthology.orchestra.run.smithi105.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 670 kB/s | 90 kB 00:00 2024-09-21T07:24:05.502 INFO:teuthology.orchestra.run.smithi105.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-09-21T07:24:05.508 INFO:teuthology.orchestra.run.smithi112.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-21T07:24:05.525 INFO:teuthology.orchestra.run.smithi112.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.6 MB/s | 26 kB 00:00 2024-09-21T07:24:05.527 INFO:teuthology.orchestra.run.smithi105.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T07:24:05.551 INFO:teuthology.orchestra.run.smithi112.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T07:24:05.552 INFO:teuthology.orchestra.run.smithi105.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T07:24:05.576 INFO:teuthology.orchestra.run.smithi112.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T07:24:05.578 INFO:teuthology.orchestra.run.smithi105.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T07:24:05.601 INFO:teuthology.orchestra.run.smithi112.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T07:24:05.603 INFO:teuthology.orchestra.run.smithi105.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T07:24:05.630 INFO:teuthology.orchestra.run.smithi105.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T07:24:05.643 INFO:teuthology.orchestra.run.smithi112.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-21T07:24:05.664 INFO:teuthology.orchestra.run.smithi105.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-09-21T07:24:05.677 INFO:teuthology.orchestra.run.smithi112.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-21T07:24:05.689 INFO:teuthology.orchestra.run.smithi105.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-21T07:24:05.702 INFO:teuthology.orchestra.run.smithi112.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:24:05.714 INFO:teuthology.orchestra.run.smithi105.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T07:24:05.727 INFO:teuthology.orchestra.run.smithi112.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T07:24:05.739 INFO:teuthology.orchestra.run.smithi105.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T07:24:05.752 INFO:teuthology.orchestra.run.smithi112.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T07:24:05.786 INFO:teuthology.orchestra.run.smithi112.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.0 MB/s | 172 kB 00:00 2024-09-21T07:24:05.807 INFO:teuthology.orchestra.run.smithi105.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-21T07:24:05.819 INFO:teuthology.orchestra.run.smithi112.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T07:24:05.845 INFO:teuthology.orchestra.run.smithi112.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 12 MB/s | 303 kB 00:00 2024-09-21T07:24:05.870 INFO:teuthology.orchestra.run.smithi112.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.5 MB/s | 220 kB 00:00 2024-09-21T07:24:05.912 INFO:teuthology.orchestra.run.smithi112.stdout:(122/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-21T07:24:05.971 INFO:teuthology.orchestra.run.smithi112.stdout:(123/157): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-09-21T07:24:05.996 INFO:teuthology.orchestra.run.smithi112.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 459 kB/s | 11 kB 00:00 2024-09-21T07:24:05.998 INFO:teuthology.orchestra.run.smithi105.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T07:24:06.021 INFO:teuthology.orchestra.run.smithi112.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 717 kB/s | 18 kB 00:00 2024-09-21T07:24:06.047 INFO:teuthology.orchestra.run.smithi112.stdout:(126/157): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-09-21T07:24:06.072 INFO:teuthology.orchestra.run.smithi112.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-21T07:24:06.097 INFO:teuthology.orchestra.run.smithi112.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T07:24:06.123 INFO:teuthology.orchestra.run.smithi112.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-21T07:24:06.148 INFO:teuthology.orchestra.run.smithi112.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T07:24:06.149 INFO:teuthology.orchestra.run.smithi105.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.7 MB/s | 3.2 MB 00:00 2024-09-21T07:24:06.173 INFO:teuthology.orchestra.run.smithi112.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T07:24:06.198 INFO:teuthology.orchestra.run.smithi112.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T07:24:06.232 INFO:teuthology.orchestra.run.smithi112.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T07:24:06.249 INFO:teuthology.orchestra.run.smithi112.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-21T07:24:06.249 INFO:teuthology.orchestra.run.smithi105.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 409 kB/s | 41 kB 00:00 2024-09-21T07:24:06.291 INFO:teuthology.orchestra.run.smithi112.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-21T07:24:06.324 INFO:teuthology.orchestra.run.smithi105.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.3 MB/s | 249 kB 00:00 2024-09-21T07:24:06.325 INFO:teuthology.orchestra.run.smithi112.stdout:(136/157): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-09-21T07:24:06.342 INFO:teuthology.orchestra.run.smithi112.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-21T07:24:06.358 INFO:teuthology.orchestra.run.smithi112.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 916 kB/s | 31 kB 00:00 2024-09-21T07:24:06.366 INFO:teuthology.orchestra.run.smithi105.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.1 MB/s | 2.3 MB 00:00 2024-09-21T07:24:06.375 INFO:teuthology.orchestra.run.smithi112.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-21T07:24:06.401 INFO:teuthology.orchestra.run.smithi112.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-21T07:24:06.426 INFO:teuthology.orchestra.run.smithi112.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-21T07:24:06.460 INFO:teuthology.orchestra.run.smithi112.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 2.2 MB/s | 74 kB 00:00 2024-09-21T07:24:06.485 INFO:teuthology.orchestra.run.smithi112.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T07:24:06.510 INFO:teuthology.orchestra.run.smithi112.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-21T07:24:06.533 INFO:teuthology.orchestra.run.smithi105.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-09-21T07:24:06.535 INFO:teuthology.orchestra.run.smithi112.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 809 kB/s | 20 kB 00:00 2024-09-21T07:24:06.577 INFO:teuthology.orchestra.run.smithi112.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-09-21T07:24:06.592 INFO:teuthology.orchestra.run.smithi105.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 1.4 MB 00:00 2024-09-21T07:24:06.602 INFO:teuthology.orchestra.run.smithi112.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T07:24:06.761 INFO:teuthology.orchestra.run.smithi112.stdout:(148/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-21T07:24:06.803 INFO:teuthology.orchestra.run.smithi112.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-09-21T07:24:06.875 INFO:teuthology.orchestra.run.smithi105.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 2.4 MB 00:00 2024-09-21T07:24:06.881 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:24:06.881 INFO:teuthology.orchestra.run.smithi105.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T07:24:06.920 INFO:teuthology.orchestra.run.smithi112.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 20 MB/s | 2.3 MB 00:00 2024-09-21T07:24:06.970 INFO:teuthology.orchestra.run.smithi112.stdout:(151/157): librados2-19.3.0-5120.g18432218.el9. 9.2 MB/s | 3.4 MB 00:00 2024-09-21T07:24:07.012 INFO:teuthology.orchestra.run.smithi112.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-09-21T07:24:07.046 INFO:teuthology.orchestra.run.smithi112.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 542 kB/s | 41 kB 00:00 2024-09-21T07:24:07.079 INFO:teuthology.orchestra.run.smithi112.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.6 MB/s | 249 kB 00:00 2024-09-21T07:24:07.130 INFO:teuthology.orchestra.run.smithi112.stdout:(155/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.6 MB/s | 3.2 MB 00:00 2024-09-21T07:24:07.172 INFO:teuthology.orchestra.run.smithi112.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-21T07:24:07.222 INFO:teuthology.orchestra.run.smithi112.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-21T07:24:07.227 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:24:07.227 INFO:teuthology.orchestra.run.smithi112.stdout:Total 20 MB/s | 228 MB 00:11 2024-09-21T07:24:08.145 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-09-21T07:24:08.298 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-09-21T07:24:08.298 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-09-21T07:24:08.476 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-09-21T07:24:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-09-21T07:24:08.628 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-09-21T07:24:10.569 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-09-21T07:24:10.569 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-09-21T07:24:10.820 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-09-21T07:24:10.820 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-09-21T07:24:17.008 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-09-21T07:24:17.113 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:24:17.134 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-09-21T07:24:17.160 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:24:17.222 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:24:17.234 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:24:17.255 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:24:17.281 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:24:17.333 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:24:17.356 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:24:17.560 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:24:17.601 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:24:17.626 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:24:17.688 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:24:17.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:24:17.898 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:24:18.025 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:24:18.031 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:24:18.049 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:24:18.124 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:24:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:24:18.154 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:24:18.184 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:24:18.269 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:24:18.294 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:24:18.364 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:24:18.368 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:24:18.398 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:24:18.426 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:24:18.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:24:18.483 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:24:18.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:24:18.594 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:24:18.622 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:24:18.646 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:24:18.674 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:24:18.691 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:24:18.715 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:24:18.739 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:24:18.785 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:24:18.809 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:24:18.871 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:24:18.893 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:24:18.923 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:24:18.935 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:24:19.095 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:24:19.114 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:24:19.213 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:24:19.315 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:24:19.329 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.406 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.441 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:24:19.546 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.625 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.643 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.758 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:24:19.811 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:24:19.846 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:24:19.847 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:24:19.935 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:24:19.955 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:24:19.979 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:24:19.994 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:24:20.012 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:24:20.039 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:24:20.055 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:24:20.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:24:20.112 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:24:20.133 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:24:20.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:24:20.206 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:24:20.210 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:24:20.257 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:24:20.271 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:24:20.290 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:24:20.302 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:24:20.318 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:24:20.346 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:24:20.386 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:24:20.430 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:24:20.439 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:24:20.465 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:24:20.471 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:24:20.506 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:24:20.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:24:20.537 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:24:20.587 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:24:20.610 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:24:20.633 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:24:20.659 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:24:20.676 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:24:20.705 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:24:20.734 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:24:20.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:24:20.820 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:24:20.865 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:24:20.894 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:24:20.920 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:24:21.025 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:24:21.080 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:24:21.439 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:24:21.524 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:24:21.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:24:21.597 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:24:21.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:24:21.643 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:24:21.685 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:24:21.692 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:24:21.722 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:24:21.773 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:24:21.819 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:24:21.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:24:21.870 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:24:21.888 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:24:21.995 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:24:22.048 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:24:22.077 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:24:22.114 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:24:22.149 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:24:22.182 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:24:22.213 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:24:22.218 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:24:22.264 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:24:22.278 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:24:22.300 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:24:22.352 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:24:22.352 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:24:22.382 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:24:22.402 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:24:22.418 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:24:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:24:22.509 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:24:22.537 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:24:22.560 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:24:22.579 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:24:22.602 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:24:22.621 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:24:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:24:22.738 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:24:22.759 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:24:22.799 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:24:22.832 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:24:22.838 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:24:22.857 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:24:22.879 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:24:22.951 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:24:22.982 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:24:23.013 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:24:23.062 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:24:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:24:23.103 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:24:23.128 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:24:23.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:24:23.264 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:24:23.293 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:24:23.326 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:24:23.354 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:24:23.366 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:24:23.398 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:24:23.402 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:24:23.427 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:24:23.445 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:24:23.462 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:24:23.481 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:24:23.504 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:24:23.519 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:24:23.551 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:24:23.554 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:24:23.581 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:24:23.588 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:24:23.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:24:23.649 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:24:23.676 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:24:23.700 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:24:23.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:24:23.736 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:24:23.749 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:24:23.824 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:24:23.852 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:24:23.867 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:24:23.906 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:24:23.909 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:24:24.017 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:24:24.074 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:24:24.636 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:24:24.707 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:24:24.741 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:24:24.781 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:24:24.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:24:24.838 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:24:24.863 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:24:24.887 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:24:24.932 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:24:24.968 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:24:25.027 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:24:25.034 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:24:25.068 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:24:25.080 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:24:25.167 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:24:25.207 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:24:25.253 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:24:25.283 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:24:25.361 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:24:25.390 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:24:25.413 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:24:25.491 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:24:26.010 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:24:26.131 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:24:26.133 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:24:26.260 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:24:27.385 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:24:27.472 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:24:27.509 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:24:27.510 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:24:27.550 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:24:27.590 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:24:27.618 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:24:27.651 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:24:27.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:24:27.921 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:24:27.983 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:24:28.013 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:24:28.031 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:24:28.043 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:24:28.079 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:24:28.121 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:24:28.160 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:24:28.197 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:24:28.577 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:24:28.608 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:24:28.680 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:24:28.694 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:24:28.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:24:28.725 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:24:28.796 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:24:28.827 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:24:30.984 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.031 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.134 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.173 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.606 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:24:31.734 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:31.760 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:24:31.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:24:31.801 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:24:31.895 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:24:31.895 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:24:31.896 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:24:31.896 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:31.936 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-09-21T07:24:43.741 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:43.915 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:24:43.931 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:24:43.931 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:43.931 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:24:43.932 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:43.968 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-09-21T07:24:43.969 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:44.139 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:24:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:24:44.156 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:44.156 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:24:44.156 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:45.535 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:24:45.562 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:45.751 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:24:45.778 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:24:45.778 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:45.778 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:24:45.778 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:24:45.779 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:24:45.779 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:45.794 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:24:45.817 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:46.010 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:24:46.034 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:46.294 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:24:46.318 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:46.424 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:24:46.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:24:46.511 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:24:46.535 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:46.536 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:24:46.641 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:24:46.698 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:24:46.719 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:24:46.761 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:24:46.769 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:24:46.846 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:24:46.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:24:46.945 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:24:46.987 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:24:47.064 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:24:47.073 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:24:47.119 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:24:47.152 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:24:47.161 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:24:47.179 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:24:47.302 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:24:47.345 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:24:47.377 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:24:47.405 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:24:47.630 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:24:47.669 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:24:47.710 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:24:47.777 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:24:47.801 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:24:47.865 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:24:47.904 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:24:47.936 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:24:48.002 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:24:48.027 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:24:50.975 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:24:51.049 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:24:51.193 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:24:51.258 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:24:52.013 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:24:52.041 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:24:52.153 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:24:52.217 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:24:52.251 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:24:52.316 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:24:52.359 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:24:52.365 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:24:52.384 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:52.435 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:24:52.447 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:24:52.542 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:24:52.545 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:24:52.569 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:24:52.593 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:24:52.593 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:52.593 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:24:52.594 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:24:52.594 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:24:52.594 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:52.603 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:24:52.636 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:24:52.648 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:24:52.747 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:24:52.816 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:24:53.563 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:24:53.596 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:24:53.619 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:24:53.619 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.619 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:24:53.620 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:24:53.620 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:24:53.620 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:53.663 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:24:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:24:53.686 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.686 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:24:53.686 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:53.739 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:24:53.762 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:53.781 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:24:53.814 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:24:53.838 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:53.881 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:24:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:24:53.905 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.905 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:24:53.905 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:53.957 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:24:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:24:53.980 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:53.980 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:24:53.980 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:24:53.981 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:24:53.981 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:54.089 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:24:54.113 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:24:54.308 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:24:54.332 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:24:58.128 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:24:58.168 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:24:58.216 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:24:58.249 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:24:58.276 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:24:58.366 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:24:58.420 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:24:58.460 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:24:58.475 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:24:58.492 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:24:58.520 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:24:58.522 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:24:58.553 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:24:58.582 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:24:58.637 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:24:58.729 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:24:58.783 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:24:58.813 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:24:58.843 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:24:58.889 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:24:59.281 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:24:59.326 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:24:59.347 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:24:59.445 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:24:59.487 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:24:59.508 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:25:00.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:25:00.272 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:25:00.322 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:25:00.334 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:25:00.413 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:25:00.425 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:25:00.450 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:25:00.475 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:25:00.479 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:25:00.501 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:25:00.501 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:25:00.584 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:25:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:25:00.585 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:25:00.620 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:25:00.646 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:25:00.669 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:25:00.671 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:25:00.672 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:25:00.694 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:25:00.700 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:00.700 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:00.756 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:25:00.756 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:25:00.840 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:25:00.865 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:25:00.871 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:00.871 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:00.887 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:00.888 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:25:00.971 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:25:01.036 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:01.059 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:25:01.059 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:25:01.143 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:25:01.208 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:25:07.233 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:25:07.234 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:25:07.235 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:25:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:25:07.246 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:25:07.247 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:25:07.248 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:25:07.249 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:25:07.250 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:25:07.251 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:25:07.252 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:25:07.253 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:25:07.254 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:25:07.255 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:25:07.256 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:25:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:25:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:25:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:25:07.257 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:25:07.725 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:07.725 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:25:07.725 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:25:07.725 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:25:07.726 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:25:07.727 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:25:07.728 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:25:07.729 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:25:07.730 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:25:07.731 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:25:07.732 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:25:07.733 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:25:07.734 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:25:07.735 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:25:07.736 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:25:07.737 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:25:07.738 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.097 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.098 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.099 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:25:10.100 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.101 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:25:10.102 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:25:10.103 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:25:10.104 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:25:10.105 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:25:10.106 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:25:10.107 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:25:10.108 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:25:10.109 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:25:10.109 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:10.109 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-09-21T07:25:10.207 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:25:10.207 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:10.207 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-09-21T07:25:10.207 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.208 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.209 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:25:10.210 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:25:10.211 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:25:10.212 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.213 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:25:10.214 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:25:10.215 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.216 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:25:10.217 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:25:10.218 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:25:10.219 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:25:10.219 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:25:10.219 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:10.219 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-09-21T07:25:10.548 DEBUG:teuthology.parallel:result is None 2024-09-21T07:25:10.612 DEBUG:teuthology.parallel:result is None 2024-09-21T07:25:10.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:25:10.763 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:25:10.785 INFO:teuthology.orchestra.run.smithi105.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:25:10.785 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:25:10.785 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:25:10.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:25:10.935 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:25:10.958 INFO:teuthology.orchestra.run.smithi112.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:25:10.958 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:25:10.958 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:25:10.960 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T07:25:10.960 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:10.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:25:10.988 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:10.988 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:25:11.026 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T07:25:11.027 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:11.027 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:25:11.054 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:25:11.120 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:11.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:25:11.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:25:11.211 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T07:25:11.211 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:11.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:25:11.236 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:25:11.300 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:11.300 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:25:11.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:25:11.389 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T07:25:11.389 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:11.389 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:25:11.413 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:25:11.478 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:11.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:25:11.502 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:25:11.567 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T07:25:11.640 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T07:25:11.640 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2024-09-21T07:25:11.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2024-09-21T07:25:11.666 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T07:25:11.667 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:25:11.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:25:11.733 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T07:25:11.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:25:11.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:25:11.804 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T07:25:11.804 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T07:25:11.804 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T07:25:11.804 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:11.804 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:25:11.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:11.961 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:11.985 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:11.986 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:25:12.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:12.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:12.140 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T07:25:12.140 INFO:tasks.ceph:config {'conf': {'client': {'client snapdir': '.customsnapfuse', 'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T07:25:12.140 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913597', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client snapdir': '.customsnapfuse', 'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'snapdirname': '.customsnapkernel', 'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDaH59tidbgaJVyprt8oI92H5nfONl8lAebahhj3a6uhipeF4PAAM9J+5ETr9jsz//vOVOIVjNE831Rhyu9KMPA=', 'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDmDpRWpKgDUEh1EJap1QVIdk+KMgjerpQ0SDXdCjS0CrzbWOKN/vGIGIM5sKr9Xwmb0a+wFHveLuHgZQRu2bw4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client snapdir': '.customsnapfuse', 'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T07:25:12.140 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:25:12.143 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:25:12.190 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:25:12.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:25:12.255 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:12.255 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:25:12.278 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:25:12.278 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-21 07:25:01.550876897 +0000 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-21 07:23:13.788667817 +0000 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-21 07:23:13.788667817 +0000 2024-09-21T07:25:12.333 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-21 07:23:13.788667817 +0000 2024-09-21T07:25:12.334 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:25:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-21T07:25:12.396 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-21T07:25:12.397 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000184734 s, 2.8 MB/s 2024-09-21T07:25:12.398 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:25:12.454 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-09-21T07:25:12.510 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:25:12.510 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:12.510 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T07:25:12.510 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:12.511 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:12.511 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-21 07:25:01.551876876 +0000 2024-09-21T07:25:12.511 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-21 07:23:14.109661108 +0000 2024-09-21T07:25:12.511 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-21 07:23:14.109661108 +0000 2024-09-21T07:25:12.511 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-21 07:23:14.109661108 +0000 2024-09-21T07:25:12.511 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:25:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-21T07:25:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-21T07:25:12.573 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000168021 s, 3.0 MB/s 2024-09-21T07:25:12.574 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:25:12.630 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-09-21T07:25:12.685 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-21 07:25:01.551876876 +0000 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-21 07:23:14.425654503 +0000 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-21 07:23:14.425654503 +0000 2024-09-21T07:25:12.686 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-21 07:23:14.425654503 +0000 2024-09-21T07:25:12.686 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:25:12.748 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-21T07:25:12.748 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-21T07:25:12.748 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000178473 s, 2.9 MB/s 2024-09-21T07:25:12.749 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:25:12.805 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-09-21T07:25:12.860 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:25:12.860 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-09-21 07:25:01.551876876 +0000 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-09-21 07:23:14.757647563 +0000 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-09-21 07:23:14.757647563 +0000 2024-09-21T07:25:12.861 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-09-21 07:23:14.757647563 +0000 2024-09-21T07:25:12.861 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:25:12.923 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-09-21T07:25:12.923 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-09-21T07:25:12.923 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.00019205 s, 2.7 MB/s 2024-09-21T07:25:12.924 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:25:12.980 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:12.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:12.981 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:25:12.998 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:25:12.998 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-09-21T07:25:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:25:13.055 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:13.055 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 07:25:01.412014735 +0000 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 07:23:13.939635339 +0000 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 07:23:13.939635339 +0000 2024-09-21T07:25:13.056 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 07:23:13.939635339 +0000 2024-09-21T07:25:13.056 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:25:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T07:25:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T07:25:13.120 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000282663 s, 1.8 MB/s 2024-09-21T07:25:13.121 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:25:13.178 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-09-21T07:25:13.234 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 07:25:01.412014735 +0000 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 07:23:14.260627012 +0000 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 07:23:14.260627012 +0000 2024-09-21T07:25:13.235 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 07:23:14.260627012 +0000 2024-09-21T07:25:13.235 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:25:13.298 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T07:25:13.299 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T07:25:13.299 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000165621 s, 3.1 MB/s 2024-09-21T07:25:13.302 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:25:13.359 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-09-21T07:25:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:25:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:13.415 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 07:25:01.413014709 +0000 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 07:23:14.594618348 +0000 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 07:23:14.594618348 +0000 2024-09-21T07:25:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 07:23:14.594618348 +0000 2024-09-21T07:25:13.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:25:13.479 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T07:25:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T07:25:13.480 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000241257 s, 2.1 MB/s 2024-09-21T07:25:13.481 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:25:13.538 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-09-21 07:25:01.413014709 +0000 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-09-21 07:23:14.926609735 +0000 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-09-21 07:23:14.926609735 +0000 2024-09-21T07:25:13.593 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-09-21 07:23:14.926609735 +0000 2024-09-21T07:25:13.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:25:13.655 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-09-21T07:25:13.656 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-09-21T07:25:13.656 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000186634 s, 2.7 MB/s 2024-09-21T07:25:13.657 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:25:13.713 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:13.713 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:25:13.713 INFO:tasks.ceph:Generating config... 2024-09-21T07:25:13.715 INFO:tasks.ceph:[client] client snapdir = .customsnapfuse 2024-09-21T07:25:13.715 INFO:tasks.ceph:[client] pid file = 2024-09-21T07:25:13.715 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T07:25:13.715 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T07:25:13.716 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T07:25:13.717 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T07:25:13.717 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T07:25:13.717 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T07:25:13.717 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T07:25:13.785 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T07:25:13.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T07:25:13.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:25:13.869 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.112'), ('mon.c', '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]')] 2024-09-21T07:25:13.869 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client snapdir': '.customsnapfuse', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T07:25:13.870 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:13.870 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:25:13.925 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.112 --addv c '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 15ee7004-a3e2-41ce-b1eb-6358972a1ae2 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = quincy 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 15ee7004-a3e2-41ce-b1eb-6358972a1ae2 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2024-09-21T07:25:14.021365+0000 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:created 2024-09-21T07:25:14.021365+0000 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 17 (quincy) 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2024-09-21T07:25:14.022 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2024-09-21T07:25:14.023 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2024-09-21T07:25:14.023 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] mon.c 2024-09-21T07:25:14.023 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T07:25:14.024 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:25:14.039 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 15ee7004-a3e2-41ce-b1eb-6358972a1ae2... 2024-09-21T07:25:14.039 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:25:14.081 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:25:14.121 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2024-09-21T07:25:14.122 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2024-09-21T07:25:14.123 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.124 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on skipped message = False 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 15ee7004-a3e2-41ce-b1eb-6358972a1ae2 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.125 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug trim objects = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.126 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 20 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 20 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 10 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:25:14.127 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2024-09-21T07:25:14.128 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:25:14.129 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: mon down mkfs grace = 300 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2024-09-21T07:25:14.130 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout: client snapdir = .customsnapfuse 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-09-21T07:25:14.131 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-09-21T07:25:14.132 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-09-21T07:25:14.133 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-09-21T07:25:14.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.105,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on skipped message = False 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 15ee7004-a3e2-41ce-b1eb-6358972a1ae2 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.135 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.136 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-09-21T07:25:14.137 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T07:25:14.138 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-09-21T07:25:14.139 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:25:14.140 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.141 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: client snapdir = .customsnapfuse 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-09-21T07:25:14.142 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T07:25:14.143 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T07:25:14.144 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T07:25:14.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T07:25:14.249 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T07:25:14.249 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:14.249 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T07:25:14.264 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:14.265 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T07:25:14.320 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:25:14.320 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:14.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:25:14.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:25:14.393 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:14.393 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:14.449 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:25:14.449 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:14.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:25:14.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:25:14.488 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:14.488 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:14.545 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T07:25:14.546 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T07:25:14.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:25:14.606 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:25:14.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:25:14.698 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:25:14.701 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T07:25:14.702 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:25:14.762 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:25:14.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T07:25:14.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:25:14.889 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:25:14.893 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T07:25:14.917 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:25:14.981 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:25:14.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T07:25:15.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:25:15.110 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:25:15.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T07:25:15.137 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T07:25:15.137 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T07:25:15.187 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T07:25:15.200 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T07:25:15.250 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T07:25:15.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T07:25:15.313 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T07:25:15.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T07:25:15.376 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T07:25:15.389 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T07:25:15.390 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:25:15.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T07:25:15.415 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:15.415 INFO:tasks.ceph:role: osd.0 2024-09-21T07:25:15.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:25:15.415 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:25:15.486 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:15.487 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:15.999 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-21T07:25:16.001 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-21T07:25:16.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T07:25:16.112 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T07:25:16.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T07:25:16.230 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:16.230 INFO:tasks.ceph:role: osd.1 2024-09-21T07:25:16.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:25:16.230 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:25:16.302 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:16.302 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:16.303 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:16.756 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-21T07:25:16.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-21T07:25:16.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T07:25:16.860 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T07:25:16.888 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T07:25:16.954 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:16.954 INFO:tasks.ceph:role: osd.2 2024-09-21T07:25:16.954 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:25:16.955 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:25:17.027 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:17.028 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:17.513 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-21T07:25:17.516 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-21T07:25:17.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T07:25:17.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T07:25:17.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T07:25:17.758 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:17.758 INFO:tasks.ceph:role: osd.3 2024-09-21T07:25:17.758 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:25:17.758 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:17.831 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:17.832 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:18.319 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2024-09-21T07:25:18.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2024-09-21T07:25:18.322 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T07:25:18.444 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T07:25:18.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:18.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:18.578+0000 7ff5749b0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T07:25:18.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:18.578+0000 7ff5749b0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T07:25:18.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:18.578+0000 7ff5749b0640 -1 bdev(0x5559f900f900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:18.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:18.578+0000 7ff5749b0640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T07:25:21.742 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T07:25:21.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:21.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:21.875+0000 7fda16bcc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T07:25:21.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:21.875+0000 7fda16bcc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T07:25:21.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:21.875+0000 7fda16bcc640 -1 bdev(0x55ff908f1900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:21.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:21.875+0000 7fda16bcc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T07:25:24.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T07:25:24.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:24.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:24.920+0000 7f9173e0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T07:25:24.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:24.920+0000 7f9173e0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T07:25:24.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:24.920+0000 7f9173e0b640 -1 bdev(0x564d45e79900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:24.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:24.920+0000 7f9173e0b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T07:25:27.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T07:25:27.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:27.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:27.968+0000 7f90e860d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T07:25:27.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:27.968+0000 7f90e860d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T07:25:27.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:27.968+0000 7f90e860d640 -1 bdev(0x55b2a141f900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:27.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:27.968+0000 7f90e860d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T07:25:31.123 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T07:25:31.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T07:25:31.181 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:31.181 INFO:tasks.ceph:role: osd.4 2024-09-21T07:25:31.181 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:25:31.181 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:31.256 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:31.257 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:31.257 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:31.257 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:31.776 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T07:25:31.778 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T07:25:31.778 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T07:25:31.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T07:25:31.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T07:25:32.004 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:32.004 INFO:tasks.ceph:role: osd.5 2024-09-21T07:25:32.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:25:32.004 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:32.076 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:32.536 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T07:25:32.538 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T07:25:32.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T07:25:32.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T07:25:32.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T07:25:32.717 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:32.717 INFO:tasks.ceph:role: osd.6 2024-09-21T07:25:32.717 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:25:32.718 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:32.790 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:32.791 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:32.791 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:32.791 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:32.791 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:33.274 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T07:25:33.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T07:25:33.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T07:25:33.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T07:25:33.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T07:25:33.513 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:25:33.513 INFO:tasks.ceph:role: osd.7 2024-09-21T07:25:33.513 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:25:33.514 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:25:33.586 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:25:33.587 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:25:33.587 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:25:33.587 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:25:33.587 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:25:34.114 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-09-21T07:25:34.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-09-21T07:25:34.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T07:25:34.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T07:25:34.302 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:34.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:34.444+0000 7f0d14c0b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T07:25:34.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:34.444+0000 7f0d14c0b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T07:25:34.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:34.444+0000 7f0d14c0b640 -1 bdev(0x558147c1d900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:34.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:34.444+0000 7f0d14c0b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T07:25:37.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T07:25:37.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:37.483+0000 7f6d0580b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T07:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:37.483+0000 7f6d0580b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T07:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:37.483+0000 7f6d0580b640 -1 bdev(0x558f2de8d900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:37.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:37.484+0000 7f6d0580b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T07:25:40.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T07:25:40.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:41.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:41.017+0000 7f9b35ceb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T07:25:41.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:41.017+0000 7f9b35ceb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T07:25:41.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:41.017+0000 7f9b35ceb640 -1 bdev(0x555c1167d900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:41.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:41.017+0000 7f9b35ceb640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T07:25:42.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:42.144 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:25:44.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T07:25:44.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:44.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:44.328+0000 7fbda462e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T07:25:44.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:44.328+0000 7fbda462e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T07:25:44.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:44.328+0000 7fbda462e640 -1 bdev(0x562642755900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T07:25:44.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:44.328+0000 7fbda462e640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T07:25:47.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T07:25:47.263 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T07:25:47.263 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.263 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T07:25:47.293 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.294 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T07:25:47.361 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.362 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T07:25:47.429 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.429 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T07:25:47.497 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.497 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T07:25:47.563 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T07:25:47.630 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:47.630 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T07:25:47.696 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T07:25:47.721 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T07:25:47.785 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.785 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T07:25:47.850 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T07:25:47.914 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T07:25:47.979 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:47.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T07:25:48.043 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:48.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T07:25:48.109 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:48.110 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T07:25:48.126 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:48.126 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T07:25:48.182 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:48.183 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T07:25:48.198 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:48.198 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T07:25:48.252 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T07:25:48.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:25:48.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:25:48.295 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDadO5maosdJBAAIyNW23hpwO32rk2iW9Eg5w== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDadO5mh0dlLRAAaYpF2t+Uk6VjVKs++AFOzw== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDadO5m6rIANRAAh2kt5k5Cl1bYkKPJ8K61CA== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDedO5m8iKKIhAAN3dzLRQ8YMY38vpHsJdxmg== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDhdO5m0jA4NBAAu3joL35rLvu39fwQ4MNxkw== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDkdO5mxUfnNhAAJ5uQUfTxnxxyDXGntIE0EA== 2024-09-21T07:25:48.296 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDndO5mXHXIORAAHmBnGC7IXT+ZSSCLGP+eEg== 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDadO5m09OYKRAAczmT4tcOHHUPZ4SNYfsyyA== 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDadO5moWd6OhAABsRGlQQJjxGagf8eheSTpw== 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mk+KIBhAAEunaRfaOoOnbVUvO+uhnMw== 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDudO5mQ8KJGhAAMvHlLrf0/OLoR/Wcde5ecA== 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2024-09-21T07:25:48.297 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDxdO5mUdbhHBAAXKKvHBseMdFfaGI/5uix2w== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD1dO5mW2olARAAHhdnmgsgu9cqwArDMFJIgQ== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQD4dO5moJCdExAACE4e5trJ+gh0HFXVPpeyvw== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mp/ImCxAASyikrNfrfPlhCoeUsnBKSA== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mg5TkDhAAe9FnO+jWhlgF35SSGFgvwA== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mMYmrEhAAgo1UsOE/3038EBINtmgPaQ== 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2024-09-21T07:25:48.298 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mxO5lFhAAjiUrrqidE0FkoM5TeXdeuw== 2024-09-21T07:25:48.315 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2024-09-21T07:25:48.315 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDadO5maosdJBAAIyNW23hpwO32rk2iW9Eg5w== 2024-09-21T07:25:48.315 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-09-21T07:25:48.315 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDadO5mh0dlLRAAaYpF2t+Uk6VjVKs++AFOzw== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDadO5m6rIANRAAh2kt5k5Cl1bYkKPJ8K61CA== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDedO5m8iKKIhAAN3dzLRQ8YMY38vpHsJdxmg== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDhdO5m0jA4NBAAu3joL35rLvu39fwQ4MNxkw== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDkdO5mxUfnNhAAJ5uQUfTxnxxyDXGntIE0EA== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDndO5mXHXIORAAHmBnGC7IXT+ZSSCLGP+eEg== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDadO5m09OYKRAAczmT4tcOHHUPZ4SNYfsyyA== 2024-09-21T07:25:48.316 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDadO5moWd6OhAABsRGlQQJjxGagf8eheSTpw== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDbdO5mk+KIBhAAEunaRfaOoOnbVUvO+uhnMw== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDudO5mQ8KJGhAAMvHlLrf0/OLoR/Wcde5ecA== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDxdO5mUdbhHBAAXKKvHBseMdFfaGI/5uix2w== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD1dO5mW2olARAAHhdnmgsgu9cqwArDMFJIgQ== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQD4dO5moJCdExAACE4e5trJ+gh0HFXVPpeyvw== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDbdO5mp/ImCxAASyikrNfrfPlhCoeUsnBKSA== 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2024-09-21T07:25:48.317 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDbdO5mg5TkDhAAe9FnO+jWhlgF35SSGFgvwA== 2024-09-21T07:25:48.318 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2024-09-21T07:25:48.318 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDbdO5mMYmrEhAAgo1UsOE/3038EBINtmgPaQ== 2024-09-21T07:25:48.318 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2024-09-21T07:25:48.318 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDbdO5mxO5lFhAAjiUrrqidE0FkoM5TeXdeuw== 2024-09-21T07:25:48.319 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:25:48.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:25:48.419 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.485 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.551 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.553 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.614 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.739 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.742 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:48.803 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:25:48.805 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:25:48.866 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.932 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:25:48.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.062 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.124 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.189 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:25:49.251 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.319 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.453 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:25:49.517 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T07:25:49.518 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T07:25:49.542 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:25:49.648 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T07:25:49.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T07:25:49.699 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:25:49.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T07:25:49.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T07:25:49.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:25:49.998 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T07:25:50.023 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:50.025 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:25:50.078 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T07:25:50.079 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T07:25:50.079 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T07:25:50.081 INFO:tasks.ceph.mon.a:Started 2024-09-21T07:25:50.082 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T07:25:50.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T07:25:50.122 INFO:tasks.ceph.mon.b:Started 2024-09-21T07:25:50.122 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T07:25:50.122 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T07:25:50.124 INFO:tasks.ceph.mon.c:Started 2024-09-21T07:25:50.125 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T07:25:50.125 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T07:25:50.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T07:25:50.128 INFO:tasks.ceph.mgr.y:Started 2024-09-21T07:25:50.129 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T07:25:50.129 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T07:25:50.131 INFO:tasks.ceph.mgr.x:Started 2024-09-21T07:25:50.131 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T07:25:50.132 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T07:25:50.135 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T07:25:50.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T07:25:50.177 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T07:25:50.178 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T07:25:50.220 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T07:25:50.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T07:25:50.263 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T07:25:50.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T07:25:50.307 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T07:25:50.307 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T07:25:50.349 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T07:25:50.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T07:25:50.392 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T07:25:50.392 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T07:25:50.436 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T07:25:50.436 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T07:25:50.478 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T07:25:50.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T07:25:50.521 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T07:25:50.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T07:25:50.565 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T07:25:50.565 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T07:25:50.608 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T07:25:50.608 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T07:25:50.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.619+0000 7f682effd640 1 -- 172.21.15.105:0/2997663909 <== mon.2 v2:172.21.15.112:3301/0 11 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v5)=0 v5) ==== 109+0+0 (secure 0 0 0) 0x7f6828014070 con 0x7f68380d04f0 2024-09-21T07:25:50.624 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T07:25:50.624 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T07:25:50.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.625+0000 7f261f263640 1 -- 172.21.15.105:0/3360463619 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f25f0005310 con 0x7f2618071010 2024-09-21T07:25:50.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.625+0000 7f682cff9640 1 -- 172.21.15.105:0/2997663909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68380d04f0 msgr2=0x7f68380b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.625+0000 7f682cff9640 1 --2- 172.21.15.105:0/2997663909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68380d04f0 0x7f68380b1870 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68280098b0 tx=0x7f6828009d70 comp rx=0 tx=0).stop 2024-09-21T07:25:50.630 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T07:25:50.630 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.626+0000 7f682cff9640 1 -- 172.21.15.105:0/2997663909 shutdown_connections 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f682cff9640 1 --2- 172.21.15.105:0/2997663909 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68380b1db0 0x7f68381efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f682cff9640 1 --2- 172.21.15.105:0/2997663909 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68380d04f0 0x7f68380b1870 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f682cff9640 1 --2- 172.21.15.105:0/2997663909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6824003a00 0x7f68381450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f682cff9640 1 -- 172.21.15.105:0/2997663909 >> 172.21.15.105:0/2997663909 conn(0x7f68380b6610 msgr2=0x7f683806cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f682cff9640 1 -- 172.21.15.105:0/2997663909 shutdown_connections 2024-09-21T07:25:50.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.628+0000 7f682cff9640 1 -- 172.21.15.105:0/2997663909 wait complete. 2024-09-21T07:25:50.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.627+0000 7f05e67fc640 1 -- 172.21.15.105:0/3243532190 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f05e804c440 con 0x7f05f00cfd00 2024-09-21T07:25:50.634 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T07:25:50.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T07:25:50.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.632+0000 7f26157fa640 1 -- 172.21.15.105:0/3360463619 <== mon.2 v2:172.21.15.112:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f260c01a7f0 con 0x7f2618071010 2024-09-21T07:25:50.644 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T07:25:50.644 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T07:25:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.636+0000 7f26157fa640 1 -- 172.21.15.105:0/3360463619 <== mon.2 v2:172.21.15.112:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f260c01ab00 con 0x7f2618071010 2024-09-21T07:25:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.639+0000 7f2602ffd640 1 -- 172.21.15.105:0/3360463619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2618071010 msgr2=0x7f26180c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.639+0000 7f2602ffd640 1 --2- 172.21.15.105:0/3360463619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2618071010 0x7f26180c44e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f260c00c310 tx=0x7f260c001e70 comp rx=0 tx=0).stop 2024-09-21T07:25:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.632+0000 7f39797fa640 1 -- 172.21.15.105:0/3521351555 <== mon.1 v2:172.21.15.112:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f397400f4f0 con 0x7f397c0cfd00 2024-09-21T07:25:50.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.636+0000 7f05e67fc640 1 -- 172.21.15.105:0/3243532190 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e804c620 con 0x7f05f00cfd00 2024-09-21T07:25:50.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 -- 172.21.15.105:0/3360463619 shutdown_connections 2024-09-21T07:25:50.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 --2- 172.21.15.105:0/3360463619 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26180cedf0 0x7f261809ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 --2- 172.21.15.105:0/3360463619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26180ca760 0x7f2618204380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 --2- 172.21.15.105:0/3360463619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2618071010 0x7f26180c44e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 -- 172.21.15.105:0/3360463619 >> 172.21.15.105:0/3360463619 conn(0x7f26180ccb90 msgr2=0x7f2618153650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 -- 172.21.15.105:0/3360463619 shutdown_connections 2024-09-21T07:25:50.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.644+0000 7f2602ffd640 1 -- 172.21.15.105:0/3360463619 wait complete. 2024-09-21T07:25:50.649 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T07:25:50.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T07:25:50.668 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T07:25:50.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T07:25:50.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.675+0000 7f612bfff640 1 -- 172.21.15.105:0/1176055468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0af480 msgr2=0x7f612c073df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.677 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.675+0000 7f612bfff640 1 --2- 172.21.15.105:0/1176055468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0af480 0x7f612c073df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6124009830 tx=0x7f612402a520 comp rx=0 tx=0).stop 2024-09-21T07:25:50.678 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T07:25:50.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T07:25:50.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.677+0000 7f612bfff640 1 -- 172.21.15.105:0/1176055468 shutdown_connections 2024-09-21T07:25:50.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.677+0000 7f612bfff640 1 --2- 172.21.15.105:0/1176055468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c0af480 0x7f612c073df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.677+0000 7f612bfff640 1 --2- 172.21.15.105:0/1176055468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0af070 0x7f612c070460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.677+0000 7f612bfff640 1 -- 172.21.15.105:0/1176055468 >> 172.21.15.105:0/1176055468 conn(0x7f612c0d1340 msgr2=0x7f612c0736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.678+0000 7f612bfff640 1 -- 172.21.15.105:0/1176055468 shutdown_connections 2024-09-21T07:25:50.690 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T07:25:50.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T07:25:50.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.682+0000 7f612bfff640 1 -- 172.21.15.105:0/1176055468 wait complete. 2024-09-21T07:25:50.693 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T07:25:50.693 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T07:25:50.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.690+0000 7f612bfff640 1 Processor -- start 2024-09-21T07:25:50.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.696+0000 7f612bfff640 1 -- start start 2024-09-21T07:25:50.701 DEBUG:tasks.ceph:set 22 configs 2024-09-21T07:25:50.701 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.698+0000 7f612bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48410/0 (socket says 172.21.15.105:48410) 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612affd640 1 -- 172.21.15.105:0/2660509079 learned_addr learned my addr 172.21.15.105:0/2660509079 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612bfff640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0af070 0x7f612c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612bfff640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c14cf60 0x7f612c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612bfff640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f612c15f5f0 con 0x7f612c074330 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612bfff640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f612c15f470 con 0x7f612c14cf60 2024-09-21T07:25:50.703 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.699+0000 7f612bfff640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f612c15ff20 con 0x7f612c0af070 2024-09-21T07:25:50.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.700+0000 7f612affd640 1 -- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0af070 msgr2=0x7f612c151e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:25:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.700+0000 7f612affd640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0af070 0x7f612c151e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.700+0000 7f612affd640 1 -- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c14cf60 msgr2=0x7f612c15dac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:25:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.700+0000 7f612affd640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c14cf60 0x7f612c15dac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.710 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.700+0000 7f612affd640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c15ca20 con 0x7f612c074330 2024-09-21T07:25:50.716 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.714+0000 7f612affd640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f611400e9f0 tx=0x7f611400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:50.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.715+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611400cbd0 con 0x7f612c074330 2024-09-21T07:25:50.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.715+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6114010430 con 0x7f612c074330 2024-09-21T07:25:50.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.715+0000 7f612bfff640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c15cd00 con 0x7f612c074330 2024-09-21T07:25:50.717 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.716+0000 7f612bfff640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c15d2c0 con 0x7f612c074330 2024-09-21T07:25:50.718 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.716+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611400ce60 con 0x7f612c074330 2024-09-21T07:25:50.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.718+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f61140108e0 con 0x7f612c074330 2024-09-21T07:25:50.719 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.718+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f61140213f0 con 0x7f612c074330 2024-09-21T07:25:50.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.720+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c0aff60 con 0x7f612c074330 2024-09-21T07:25:50.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.724+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6114010bd0 con 0x7f612c074330 2024-09-21T07:25:50.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.804+0000 7fe2b3a69640 1 -- 172.21.15.105:0/3285054798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac0ca970 msgr2=0x7fe2ac0af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.804+0000 7fe2b3a69640 1 --2- 172.21.15.105:0/3285054798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac0af660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0cdf80 tx=0x7fe29c02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.808+0000 7fe2b3a69640 1 -- 172.21.15.105:0/3285054798 shutdown_connections 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.808+0000 7fe2b3a69640 1 --2- 172.21.15.105:0/3285054798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 0x7fe2ac162470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.808+0000 7fe2b3a69640 1 --2- 172.21.15.105:0/3285054798 >> v2:172.21.15.112:3300/0 conn(0x7fe2ac15ee80 0x7fe2ac15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.808+0000 7fe2b3a69640 1 --2- 172.21.15.105:0/3285054798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac0af660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.808+0000 7fe2b3a69640 1 -- 172.21.15.105:0/3285054798 >> 172.21.15.105:0/3285054798 conn(0x7fe2ac0ccb90 msgr2=0x7fe2ac0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.824+0000 7fe2b3a69640 1 -- 172.21.15.105:0/3285054798 shutdown_connections 2024-09-21T07:25:50.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.824+0000 7fe2b3a69640 1 -- 172.21.15.105:0/3285054798 wait complete. 2024-09-21T07:25:50.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.827+0000 7fe2b3a69640 1 Processor -- start 2024-09-21T07:25:50.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.827+0000 7fe2b3a69640 1 -- start start 2024-09-21T07:25:50.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.830+0000 7fe2b3a69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac1fc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.830+0000 7fe2b3a69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 0x7fe2ac2063e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.830+0000 7fe2b1805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac1fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.830+0000 7fe2b1805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac1fc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:57124/0 (socket says 172.21.15.105:57124) 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b1805640 1 -- 172.21.15.105:0/4255119969 learned_addr learned my addr 172.21.15.105:0/4255119969 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b3a69640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 0x7fe2ac0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b3a69640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac162b30 con 0x7fe2ac15ee80 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b2006640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 0x7fe2ac0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b3a69640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac1629b0 con 0x7fe2ac0ca970 2024-09-21T07:25:50.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.831+0000 7fe2b3a69640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac162cb0 con 0x7fe2ac15f7d0 2024-09-21T07:25:50.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.832+0000 7fe2b1004640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 0x7fe2ac2063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.833+0000 7fe2b1004640 1 -- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 msgr2=0x7fe2ac0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.833+0000 7fe2b1004640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 0x7fe2ac0ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.833+0000 7fe2b1004640 1 -- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 msgr2=0x7fe2ac1fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.833+0000 7fe2b1004640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac1fc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.833+0000 7fe2b1004640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac09aa20 con 0x7fe2ac15ee80 2024-09-21T07:25:50.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.834+0000 7fe2b1004640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 0x7fe2ac2063e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800b410 tx=0x7fe2a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.839+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8015020 con 0x7fe2ac15ee80 2024-09-21T07:25:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.839+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a8011940 con 0x7fe2ac15ee80 2024-09-21T07:25:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.839+0000 7fe2b3a69640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac206920 con 0x7fe2ac15ee80 2024-09-21T07:25:50.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.839+0000 7fe2b3a69640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac202cd0 con 0x7fe2ac15ee80 2024-09-21T07:25:50.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.841+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8011d90 con 0x7fe2ac15ee80 2024-09-21T07:25:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.842+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe2a8018020 con 0x7fe2ac15ee80 2024-09-21T07:25:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.843+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe2a801a5f0 con 0x7fe2ac15ee80 2024-09-21T07:25:50.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.843+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c005290 con 0x7fe2ac15ee80 2024-09-21T07:25:50.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.850+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe2a801f070 con 0x7fe2ac15ee80 2024-09-21T07:25:50.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.882+0000 7fc422227640 1 -- 172.21.15.105:0/512540542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc414005ea0 msgr2=0x7fc4140062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.882+0000 7fc422227640 1 --2- 172.21.15.105:0/512540542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc414005ea0 0x7fc4140062b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc41404b7f0 tx=0x7fc41002a580 comp rx=0 tx=0).stop 2024-09-21T07:25:50.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7f398324d640 1 -- 172.21.15.105:0/3521351555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f397c0b6300 con 0x7f397c0cfd00 2024-09-21T07:25:50.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7fc422227640 1 -- 172.21.15.105:0/512540542 shutdown_connections 2024-09-21T07:25:50.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7fc422227640 1 --2- 172.21.15.105:0/512540542 >> v2:172.21.15.112:3300/0 conn(0x7fc414034a40 0x7fc4140e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7fc422227640 1 --2- 172.21.15.105:0/512540542 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc4140340f0 0x7fc414034500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7fc422227640 1 --2- 172.21.15.105:0/512540542 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc414005ea0 0x7fc4140062b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.883+0000 7fc422227640 1 -- 172.21.15.105:0/512540542 >> 172.21.15.105:0/512540542 conn(0x7fc41401d9a0 msgr2=0x7fc414039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.885+0000 7fc422227640 1 -- 172.21.15.105:0/512540542 shutdown_connections 2024-09-21T07:25:50.891 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.888+0000 7fc422227640 1 -- 172.21.15.105:0/512540542 wait complete. 2024-09-21T07:25:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.893+0000 7f39797fa640 1 -- 172.21.15.105:0/3521351555 <== mon.1 v2:172.21.15.112:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f397c13d730 con 0x7f397c0cfd00 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 -- 172.21.15.105:0/3521351555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f397c0cfd00 msgr2=0x7f397c159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 --2- 172.21.15.105:0/3521351555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f397c0cfd00 0x7f397c159ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f397400ed70 tx=0x7f397400c6e0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 -- 172.21.15.105:0/3521351555 shutdown_connections 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 --2- 172.21.15.105:0/3521351555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f397c0cfd00 0x7f397c159ec0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 --2- 172.21.15.105:0/3521351555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f397c0b4e70 0x7f397c1477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 --2- 172.21.15.105:0/3521351555 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f397c070930 0x7f397c13d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 -- 172.21.15.105:0/3521351555 >> 172.21.15.105:0/3521351555 conn(0x7f397c06cf80 msgr2=0x7f397c0b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 -- 172.21.15.105:0/3521351555 shutdown_connections 2024-09-21T07:25:50.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.895+0000 7f3966ffd640 1 -- 172.21.15.105:0/3521351555 wait complete. 2024-09-21T07:25:50.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.897+0000 7fc422227640 1 Processor -- start 2024-09-21T07:25:50.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.903+0000 7fc422227640 1 -- start start 2024-09-21T07:25:50.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414005ea0 0x7fc4140ec760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 0x7fc4140ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc414033b50 con 0x7fc4140340f0 2024-09-21T07:25:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc414007600 con 0x7fc414005ea0 2024-09-21T07:25:50.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.905+0000 7fc422227640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc414006ec0 con 0x7fc414034a40 2024-09-21T07:25:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.906+0000 7fc41bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.906+0000 7fc41bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:48530/0 (socket says 172.21.15.105:48530) 2024-09-21T07:25:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.906+0000 7fc41bfff640 1 -- 172.21.15.105:0/1808814801 learned_addr learned my addr 172.21.15.105:0/1808814801 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.906+0000 7fc41affd640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 0x7fc4140ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.906+0000 7fc41b7fe640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414005ea0 0x7fc4140ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:50.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 -- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 msgr2=0x7fc4140de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 -- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414005ea0 msgr2=0x7fc4140ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:50.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414005ea0 0x7fc4140ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:50.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 -- 172.21.15.105:0/1808814801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140d6720 con 0x7fc4140340f0 2024-09-21T07:25:50.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41bfff640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:25:50.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.907+0000 7fc41affd640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 0x7fc4140ddc80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d6b0 tx=0x7fc40800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:50.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.908+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408017070 con 0x7fc4140340f0 2024-09-21T07:25:50.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.908+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc408005310 con 0x7fc4140340f0 2024-09-21T07:25:50.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.908+0000 7fc422227640 1 -- 172.21.15.105:0/1808814801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4140d7eb0 con 0x7fc4140340f0 2024-09-21T07:25:50.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.908+0000 7fc422227640 1 -- 172.21.15.105:0/1808814801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4140d7120 con 0x7fc4140340f0 2024-09-21T07:25:50.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.912+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408009820 con 0x7fc4140340f0 2024-09-21T07:25:50.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.912+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4080099c0 con 0x7fc4140340f0 2024-09-21T07:25:50.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.912+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4080105e0 con 0x7fc4140340f0 2024-09-21T07:25:50.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.915+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc005290 con 0x7fc4140340f0 2024-09-21T07:25:50.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.919+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc4080047a0 con 0x7fc4140340f0 2024-09-21T07:25:50.934 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi105') 2024-09-21T07:25:50.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.984+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f05d80016f0 con 0x7f05f00cfd00 2024-09-21T07:25:50.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.993+0000 7f05e67fc640 1 -- 172.21.15.105:0/3243532190 <== mon.0 v2:172.21.15.105:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f05e804c440 con 0x7f05f00cfd00 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f00cfd00 msgr2=0x7f05f014f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 --2- 172.21.15.105:0/3243532190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f00cfd00 0x7f05f014f910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05e800aea0 tx=0x7f05e8004a40 comp rx=0 tx=0).stop 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 shutdown_connections 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 --2- 172.21.15.105:0/3243532190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05f00b4e70 0x7f05f0145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 --2- 172.21.15.105:0/3243532190 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05f00cfd00 0x7f05f014f910 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 --2- 172.21.15.105:0/3243532190 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05f0070930 0x7f05f014ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:50.999+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 >> 172.21.15.105:0/3243532190 conn(0x7f05f006cf80 msgr2=0x7f05f006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.000+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 shutdown_connections 2024-09-21T07:25:51.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.000+0000 7f05f7d33640 1 -- 172.21.15.105:0/3243532190 wait complete. 2024-09-21T07:25:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.002+0000 7f66a6121640 1 -- 172.21.15.105:0/1905285904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 msgr2=0x7f66a00ce0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.002+0000 7f66a6121640 1 --2- 172.21.15.105:0/1905285904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a00ce0f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6698005fd0 tx=0x7f669802a8d0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.012+0000 7f3066989640 1 -- 172.21.15.105:0/2410785114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3048004800 msgr2=0x7f3048004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.012+0000 7f3066989640 1 --2- 172.21.15.105:0/2410785114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3048004800 0x7f3048004c50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3050001aa0 tx=0x7f3048005830 comp rx=0 tx=0).stop 2024-09-21T07:25:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.012+0000 7f3066989640 1 -- 172.21.15.105:0/2410785114 shutdown_connections 2024-09-21T07:25:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.012+0000 7f3066989640 1 --2- 172.21.15.105:0/2410785114 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3048004800 0x7f3048004c50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.012+0000 7f3066989640 1 -- 172.21.15.105:0/2410785114 >> 172.21.15.105:0/2410785114 conn(0x7f30600cff40 msgr2=0x7f30600d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.013+0000 7f3066989640 1 -- 172.21.15.105:0/2410785114 shutdown_connections 2024-09-21T07:25:51.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.013+0000 7f3066989640 1 -- 172.21.15.105:0/2410785114 wait complete. 2024-09-21T07:25:51.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.015+0000 7f66a6121640 1 -- 172.21.15.105:0/1905285904 shutdown_connections 2024-09-21T07:25:51.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.015+0000 7f66a6121640 1 --2- 172.21.15.105:0/1905285904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a00ce0f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.015+0000 7f66a6121640 1 --2- 172.21.15.105:0/1905285904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 0x7f66a00c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.015+0000 7f66a6121640 1 -- 172.21.15.105:0/1905285904 >> 172.21.15.105:0/1905285904 conn(0x7f66a00ccb90 msgr2=0x7f66a00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.016+0000 7f3066989640 1 Processor -- start 2024-09-21T07:25:51.034 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.030+0000 7f66a6121640 1 -- 172.21.15.105:0/1905285904 shutdown_connections 2024-09-21T07:25:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.030+0000 7f66a6121640 1 -- 172.21.15.105:0/1905285904 wait complete. 2024-09-21T07:25:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.033+0000 7f66a6121640 1 Processor -- start 2024-09-21T07:25:51.035 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.033+0000 7f66a6121640 1 -- start start 2024-09-21T07:25:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.032+0000 7f3066989640 1 -- start start 2024-09-21T07:25:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.034+0000 7f3066989640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3060215720 0x7f306016a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.034+0000 7f3066989640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 0x7f306016ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.034+0000 7f3066989640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 0x7f30602164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.034+0000 7f3066989640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30600cdb10 con 0x7f3060215b30 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.034+0000 7f305f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 0x7f306016ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f305f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 0x7f306016ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48486/0 (socket says 172.21.15.105:48486) 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f305f7fe640 1 -- 172.21.15.105:0/3211135238 learned_addr learned my addr 172.21.15.105:0/3211135238 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3066989640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3060075000 con 0x7f3060216090 2024-09-21T07:25:51.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3066989640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f30600cdc90 con 0x7f3060215720 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 0x7f30602164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 -- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3060215720 msgr2=0x7f306016a860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3060215720 0x7f306016a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 -- 172.21.15.105:0/3211135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 msgr2=0x7f306016ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 0x7f306016ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30602169e0 con 0x7f3060216090 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f3064f26640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 0x7f30602164a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30600cda80 tx=0x7f30540079c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054017020 con 0x7f3060216090 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3054013830 con 0x7f3060216090 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054013c80 con 0x7f3060216090 2024-09-21T07:25:51.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f3066989640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3060216ef0 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f3066989640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f306016b6b0 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.038+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f305401a020 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.038+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f305401b3b0 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.039+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3040005290 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3054021030 con 0x7f3060216090 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.035+0000 7f66a6121640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f66a6121640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 0x7f66a0074b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f66a6121640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a0216400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48496/0 (socket says 172.21.15.105:48496) 2024-09-21T07:25:51.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669f7fe640 1 -- 172.21.15.105:0/3872424006 learned_addr learned my addr 172.21.15.105:0/3872424006 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669effd640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 0x7f66a0074b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f66a6121640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0070550 con 0x7f66a006f9a0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f66a6121640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66a00703d0 con 0x7f66a00cdce0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f66a6121640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f66a00ce630 con 0x7f66a00c30e0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669ffff640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a0216400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669f7fe640 1 -- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 msgr2=0x7f66a0074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.036+0000 7f669f7fe640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 0x7f66a0074b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f669f7fe640 1 -- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 msgr2=0x7f66a0216400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f669f7fe640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a0216400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f669f7fe640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a0214980 con 0x7f66a006f9a0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f669f7fe640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f669000c370 tx=0x7f669000c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690017070 con 0x7f66a006f9a0 2024-09-21T07:25:51.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66900035f0 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f66a6121640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a0217640 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.038+0000 7f66a6121640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a00c7410 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.039+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66900039e0 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.040+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f669001a070 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.040+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6690012740 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.041+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6680005290 con 0x7f66a006f9a0 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 -- 172.21.15.105:0/738076840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40ce050 msgr2=0x7f4ec4072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 --2- 172.21.15.105:0/738076840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40ce050 0x7f4ec4072d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc005fd0 tx=0x7f4ebc02ac20 comp rx=0 tx=0).stop 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 -- 172.21.15.105:0/738076840 shutdown_connections 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 --2- 172.21.15.105:0/738076840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec40cf9d0 0x7f4ec40cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 --2- 172.21.15.105:0/738076840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40ce050 0x7f4ec4072d60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 -- 172.21.15.105:0/738076840 >> 172.21.15.105:0/738076840 conn(0x7f4ec40cff40 msgr2=0x7f4ec40d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f66a00c3a30 con 0x7f66a006f9a0 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 -- 172.21.15.105:0/855792285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 msgr2=0x7f14140af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 --2- 172.21.15.105:0/855792285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 0x7f14140af3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f14140ce470 tx=0x7f140002a620 comp rx=0 tx=0).stop 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 -- 172.21.15.105:0/855792285 shutdown_connections 2024-09-21T07:25:51.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 --2- 172.21.15.105:0/855792285 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 0x7f14140af3b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 --2- 172.21.15.105:0/855792285 >> v2:172.21.15.112:3300/0 conn(0x7f14140ca970 0x7f141406f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f1418bc6640 1 -- 172.21.15.105:0/855792285 >> 172.21.15.105:0/855792285 conn(0x7f14140ccb90 msgr2=0x7f14140ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.037+0000 7f4eca8f9640 1 -- 172.21.15.105:0/738076840 shutdown_connections 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.038+0000 7f4eca8f9640 1 -- 172.21.15.105:0/738076840 wait complete. 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.042+0000 7f4eca8f9640 1 Processor -- start 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.043+0000 7f4eca8f9640 1 -- start start 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec40cf5c0 0x7f4ec41fde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec40cf9d0 0x7f4ec41687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4074f70 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec4074df0 con 0x7f4ec40cf9d0 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4eca8f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40732a0 con 0x7f4ec40cf5c0 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4ec8e96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4ec8e96640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48506/0 (socket says 172.21.15.105:48506) 2024-09-21T07:25:51.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.045+0000 7f4ec8e96640 1 -- 172.21.15.105:0/4255628283 learned_addr learned my addr 172.21.15.105:0/4255628283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec37fe640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec40cf9d0 0x7f4ec41687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 -- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec40cf5c0 msgr2=0x7f4ec41fde30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:25:51.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec40cf5c0 0x7f4ec41fde30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 -- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec40cf9d0 msgr2=0x7f4ec41687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec40cf9d0 0x7f4ec41687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 -- 172.21.15.105:0/4255628283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41ffbb0 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.046+0000 7f4ec8e96640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009e50 tx=0x7f4ebc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.055+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc041020 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f1418bc6640 1 -- 172.21.15.105:0/855792285 shutdown_connections 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.055+0000 7f4eca8f9640 1 -- 172.21.15.105:0/4255628283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec40c6130 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.055+0000 7f4eca8f9640 1 -- 172.21.15.105:0/4255628283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec40c66f0 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ebc0046b0 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc038040 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ebc044020 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ebc03d410 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.060+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea4005290 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.060+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f612c0af480 con 0x7f612c074330 2024-09-21T07:25:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.058+0000 7f1418bc6640 1 -- 172.21.15.105:0/855792285 wait complete. 2024-09-21T07:25:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.061+0000 7f1418bc6640 1 Processor -- start 2024-09-21T07:25:51.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.061+0000 7f1418bc6640 1 -- start start 2024-09-21T07:25:51.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.063+0000 7f1418bc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 0x7f14140b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.066+0000 7f1418bc6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 0x7f14142053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.066+0000 7f1418bc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 0x7f1414205930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.068 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.066+0000 7f1411d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 0x7f14142053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.066+0000 7f1411d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 0x7f14142053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:48588/0 (socket says 172.21.15.105:48588) 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.066+0000 7f1411d9b640 1 -- 172.21.15.105:0/3615064275 learned_addr learned my addr 172.21.15.105:0/3615064275 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.067+0000 7f1412d9d640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 0x7f1414205930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.067+0000 7f1418bc6640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f14140ce900 con 0x7f14140aefa0 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.067+0000 7f1418bc6640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14140ce780 con 0x7f1414162470 2024-09-21T07:25:51.069 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.067+0000 7f1418bc6640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f14140cea80 con 0x7f14140ca970 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 -- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 msgr2=0x7f14142053f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 0x7f14142053f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 -- 172.21.15.105:0/3615064275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 msgr2=0x7f14140b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 0x7f14140b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14140b5950 con 0x7f1414162470 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.068+0000 7f1412d9d640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 0x7f1414205930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1400031000 tx=0x7f1400003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.069+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140003e070 con 0x7f1414162470 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.069+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14000049c0 con 0x7f1414162470 2024-09-21T07:25:51.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.069+0000 7f1418bc6640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1414206170 con 0x7f1414162470 2024-09-21T07:25:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.069+0000 7f1418bc6640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141409b120 con 0x7f1414162470 2024-09-21T07:25:51.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.070+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400035040 con 0x7f1414162470 2024-09-21T07:25:51.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.072+0000 7f61237fe640 1 -- 172.21.15.105:0/2660509079 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7f6114010db0 con 0x7f612c074330 2024-09-21T07:25:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.072+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1400041070 con 0x7f1414162470 2024-09-21T07:25:51.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.072+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4ebc04b030 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.073+0000 7f63e82bd640 1 -- 172.21.15.105:0/1094223681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e00cfd00 msgr2=0x7f63e00d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.073+0000 7f63e82bd640 1 --2- 172.21.15.105:0/1094223681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e00cfd00 0x7f63e00d0110 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005fd0 tx=0x7f63dc02b880 comp rx=0 tx=0).stop 2024-09-21T07:25:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 msgr2=0x7f612c14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f611400e9f0 tx=0x7f611400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 shutdown_connections 2024-09-21T07:25:51.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f612c14cf60 0x7f612c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f612c0af070 0x7f612c151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 --2- 172.21.15.105:0/2660509079 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f612c074330 0x7f612c14e160 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 >> 172.21.15.105:0/2660509079 conn(0x7f612c0d1340 msgr2=0x7f612c0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 shutdown_connections 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f61217fa640 1 -- 172.21.15.105:0/2660509079 wait complete. 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1400031740 con 0x7f1414162470 2024-09-21T07:25:51.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.079+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14140c3840 con 0x7f1414162470 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.077+0000 7f63e82bd640 1 -- 172.21.15.105:0/1094223681 shutdown_connections 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.077+0000 7f63e82bd640 1 --2- 172.21.15.105:0/1094223681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e00cfd00 0x7f63e00d0110 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.077+0000 7f63e82bd640 1 --2- 172.21.15.105:0/1094223681 >> v2:172.21.15.112:3300/0 conn(0x7f63e0070930 0x7f63e0070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.077+0000 7f63e82bd640 1 -- 172.21.15.105:0/1094223681 >> 172.21.15.105:0/1094223681 conn(0x7f63e006cf80 msgr2=0x7f63e006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.078+0000 7f63e82bd640 1 -- 172.21.15.105:0/1094223681 shutdown_connections 2024-09-21T07:25:51.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.078+0000 7f63e82bd640 1 -- 172.21.15.105:0/1094223681 wait complete. 2024-09-21T07:25:51.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.083+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f14140b5950 con 0x7f1414162470 2024-09-21T07:25:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.080+0000 7f63e82bd640 1 Processor -- start 2024-09-21T07:25:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.084+0000 7f63e82bd640 1 -- start start 2024-09-21T07:25:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 -- 172.21.15.105:0/1649097907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a4070890 msgr2=0x7f43a41663d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 --2- 172.21.15.105:0/1649097907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a4070890 0x7f43a41663d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4394009860 tx=0x7f439402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f63e82bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e82bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e00b4e70 0x7f63e014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e6059640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e6059640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:48616/0 (socket says 172.21.15.105:48616) 2024-09-21T07:25:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e6059640 1 -- 172.21.15.105:0/3273258172 learned_addr learned my addr 172.21.15.105:0/3273258172 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e82bd640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e00cfd00 0x7f63e0150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e5858640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e00b4e70 0x7f63e014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00b5400 con 0x7f63e00cfd00 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63e00b5280 con 0x7f63e00b4e70 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f63e015e990 con 0x7f63e0070930 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.088+0000 7f63e685a640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e00cfd00 0x7f63e0150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f63e6059640 1 -- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e00b4e70 msgr2=0x7f63e014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f63e6059640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e00b4e70 0x7f63e014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f63e6059640 1 -- 172.21.15.105:0/3273258172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e00cfd00 msgr2=0x7f63e0150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f63e6059640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e00cfd00 0x7f63e0150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f63e6059640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e014cad0 con 0x7f63e0070930 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 -- 172.21.15.105:0/1649097907 shutdown_connections 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 --2- 172.21.15.105:0/1649097907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a4070890 0x7f43a41663d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 -- 172.21.15.105:0/1649097907 >> 172.21.15.105:0/1649097907 conn(0x7f43a40d0910 msgr2=0x7f43a40d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.086+0000 7f43abbc6640 1 -- 172.21.15.105:0/1649097907 shutdown_connections 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.087+0000 7f43abbc6640 1 -- 172.21.15.105:0/1649097907 wait complete. 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.089+0000 7f43abbc6640 1 Processor -- start 2024-09-21T07:25:51.093 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.092+0000 7f56e2920640 1 -- 172.21.15.105:0/4110108917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4036b40 msgr2=0x7f56d4006660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.092+0000 7f56e2920640 1 --2- 172.21.15.105:0/4110108917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4036b40 0x7f56d4006660 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f56d400ca70 tx=0x7f56c802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.090+0000 7f43abbc6640 1 -- start start 2024-09-21T07:25:51.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.092+0000 7f63e6059640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b3e0 tx=0x7f63d000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.092+0000 7f43abbc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a4070890 0x7f43a4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43abbc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43abbc6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 0x7f43a40a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43a9161640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43a9161640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48538/0 (socket says 172.21.15.105:48538) 2024-09-21T07:25:51.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43a9161640 1 -- 172.21.15.105:0/33951962 learned_addr learned my addr 172.21.15.105:0/33951962 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43abbc6640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406d270 con 0x7f43a40affd0 2024-09-21T07:25:51.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43abbc6640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f43a406d0f0 con 0x7f43a4070890 2024-09-21T07:25:51.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.093+0000 7f43abbc6640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f43a406d3f0 con 0x7f43a40d00b0 2024-09-21T07:25:51.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.095+0000 7f43aa163640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 0x7f43a40a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.095+0000 7f43a9962640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a4070890 0x7f43a4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.097+0000 7f43aa163640 1 -- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a4070890 msgr2=0x7f43a4155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.097+0000 7f43aa163640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a4070890 0x7f43a4155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.097+0000 7f43aa163640 1 -- 172.21.15.105:0/33951962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 msgr2=0x7f43a40a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f43aa163640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f43aa163640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a41575e0 con 0x7f43a40d00b0 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f43a9161640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0015020 con 0x7f63e0070930 2024-09-21T07:25:51.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 -- 172.21.15.105:0/4110108917 shutdown_connections 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 --2- 172.21.15.105:0/4110108917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4036b40 0x7f56d4006660 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 --2- 172.21.15.105:0/4110108917 >> v2:172.21.15.112:3300/0 conn(0x7f56d4009ba0 0x7f56d40df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 -- 172.21.15.105:0/4110108917 >> 172.21.15.105:0/4110108917 conn(0x7f56d4051420 msgr2=0x7f56d4051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d0011940 con 0x7f63e0070930 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e014c630 con 0x7f63e0070930 2024-09-21T07:25:51.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e00c5c60 con 0x7f63e0070930 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011d90 con 0x7f63e0070930 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 -- 172.21.15.105:0/4110108917 shutdown_connections 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f56e2920640 1 -- 172.21.15.105:0/4110108917 wait complete. 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f7f86464640 1 -- 172.21.15.105:0/2755202564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f80070930 msgr2=0x7f7f80070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f7f86464640 1 --2- 172.21.15.105:0/2755202564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f80070930 0x7f7f80070d40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009830 tx=0x7f7f7002a510 comp rx=0 tx=0).stop 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.099+0000 7f7f86464640 1 -- 172.21.15.105:0/2755202564 shutdown_connections 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.099+0000 7f7f86464640 1 --2- 172.21.15.105:0/2755202564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f80070930 0x7f7f80070d40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.099+0000 7f7f86464640 1 -- 172.21.15.105:0/2755202564 >> 172.21.15.105:0/2755202564 conn(0x7f7f8006cf80 msgr2=0x7f7f8006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.099+0000 7f7f86464640 1 -- 172.21.15.105:0/2755202564 shutdown_connections 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 -- 172.21.15.105:0/4121353762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7000 msgr2=0x7f9768005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 --2- 172.21.15.105:0/4121353762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7000 0x7f9768005de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9768034920 tx=0x7f975c02a580 comp rx=0 tx=0).stop 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 -- 172.21.15.105:0/4121353762 shutdown_connections 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 --2- 172.21.15.105:0/4121353762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7000 0x7f9768005de0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9768034920 tx=0x7f975c02a580 comp rx=0 tx=0).stop 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 --2- 172.21.15.105:0/4121353762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768052360 0x7f97680e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 -- 172.21.15.105:0/4121353762 >> 172.21.15.105:0/4121353762 conn(0x7f976801d9a0 msgr2=0x7f9768039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.096+0000 7f9776d24640 1 -- 172.21.15.105:0/4121353762 shutdown_connections 2024-09-21T07:25:51.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.098+0000 7f43aa163640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 0x7f43a40a99c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4394031000 tx=0x7f4394033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.097+0000 7f9776d24640 1 -- 172.21.15.105:0/4121353762 wait complete. 2024-09-21T07:25:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.100+0000 7f9776d24640 1 Processor -- start 2024-09-21T07:25:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.100+0000 7f9776d24640 1 -- start start 2024-09-21T07:25:51.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.101+0000 7f56e2920640 1 Processor -- start 2024-09-21T07:25:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.099+0000 7f7f86464640 1 -- 172.21.15.105:0/2755202564 wait complete. 2024-09-21T07:25:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f56e2920640 1 -- start start 2024-09-21T07:25:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7fca417c6640 1 -- 172.21.15.105:0/2883519565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca34008850 msgr2=0x7fca340e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7fca417c6640 1 --2- 172.21.15.105:0/2883519565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca34008850 0x7fca340e6e20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009830 tx=0x7fca2c02a540 comp rx=0 tx=0).stop 2024-09-21T07:25:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f9776d24640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 0x7f97680d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f9776d24640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 0x7f9768053520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f9774ac0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 0x7f97680d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f9774ac0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 0x7f97680d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:57256/0 (socket says 172.21.15.105:57256) 2024-09-21T07:25:51.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.102+0000 7f9774ac0640 1 -- 172.21.15.105:0/415110746 learned_addr learned my addr 172.21.15.105:0/415110746 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 -- 172.21.15.105:0/2883519565 shutdown_connections 2024-09-21T07:25:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 --2- 172.21.15.105:0/2883519565 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 0x7fca34055f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 --2- 172.21.15.105:0/2883519565 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca34008850 0x7fca340e6e20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 -- 172.21.15.105:0/2883519565 >> 172.21.15.105:0/2883519565 conn(0x7fca3401eab0 msgr2=0x7fca3403a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 -- 172.21.15.105:0/2883519565 shutdown_connections 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f9776d24640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768053a60 0x7f97680e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f97680512e0 con 0x7f97680e7410 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f976ffff640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 0x7f9768053520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f97752c1640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768053a60 0x7f97680e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97680e3620 con 0x7f9768052360 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f9768051460 con 0x7f9768053a60 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 -- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768053a60 msgr2=0x7f97680e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768053a60 0x7f97680e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 -- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 msgr2=0x7f97680d9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 0x7f97680d9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9768049330 con 0x7f97680e7410 2024-09-21T07:25:51.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f976ffff640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 0x7f9768053520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9768052310 tx=0x7f975c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c03f070 con 0x7f97680e7410 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f975c005310 con 0x7f97680e7410 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97680d9090 con 0x7f97680e7410 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97680d9650 con 0x7f97680e7410 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.107+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c003800 con 0x7f97680e7410 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f56e2920640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56d4005c60 0x7f56d4021d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f56e2920640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f56e2920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56d40dd360 0x7f56d40dd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f56e2920640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4006ba0 con 0x7f56d4009ba0 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56db7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48558/0 (socket says 172.21.15.105:48558) 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56db7fe640 1 -- 172.21.15.105:0/1998042106 learned_addr learned my addr 172.21.15.105:0/1998042106 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56e0ebd640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56d40dd360 0x7f56d40dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56e2920640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f56d400be40 con 0x7f56d40dd360 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56e2920640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f56d4006d20 con 0x7f56d4005c60 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f56dbfff640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56d4005c60 0x7f56d4021d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e00c5ec0 con 0x7f63e0070930 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63d0018020 con 0x7f63e0070930 2024-09-21T07:25:51.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.104+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63d0019a70 con 0x7f63e0070930 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394031b50 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4394031e00 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f43abbc6640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a4157ae0 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.103+0000 7f43abbc6640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a4157dc0 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.105+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394005280 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4394004b30 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f43940035f0 con 0x7f43a40d00b0 2024-09-21T07:25:51.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.108+0000 7f7f86464640 1 Processor -- start 2024-09-21T07:25:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.109+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f63d0019e50 con 0x7f63e0070930 2024-09-21T07:25:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.108+0000 7f7f86464640 1 -- start start 2024-09-21T07:25:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.110+0000 7f7f86464640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f975c0039a0 con 0x7f97680e7410 2024-09-21T07:25:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f975c043410 con 0x7f97680e7410 2024-09-21T07:25:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f7ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:48680/0 (socket says 172.21.15.105:48680) 2024-09-21T07:25:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f86464640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f800b4e70 0x7f7f80156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f7ffff640 1 -- 172.21.15.105:0/1133331144 learned_addr learned my addr 172.21.15.105:0/1133331144 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.106+0000 7fca417c6640 1 -- 172.21.15.105:0/2883519565 wait complete. 2024-09-21T07:25:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f86464640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 0x7f7f80149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7f800b5400 con 0x7f7f800cfd00 2024-09-21T07:25:51.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.109+0000 7fca417c6640 1 Processor -- start 2024-09-21T07:25:51.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7f7f84a01640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 0x7f7f80149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.111+0000 7fca417c6640 1 -- start start 2024-09-21T07:25:51.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f800b5280 con 0x7f7f800b4e70 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8015e990 con 0x7f7f80070930 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fe28c005580 con 0x7fe2ac15ee80 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f56db7fe640 1 -- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56d4005c60 msgr2=0x7f56d4021d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f56db7fe640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56d4005c60 0x7f56d4021d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f56db7fe640 1 -- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56d40dd360 msgr2=0x7f56d40dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f56db7fe640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56d40dd360 0x7f56d40dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f56db7fe640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d40ddcb0 con 0x7f56d4009ba0 2024-09-21T07:25:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.112+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4384005290 con 0x7f43a40d00b0 2024-09-21T07:25:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f56db7fe640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c7a0 tx=0x7f56d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 -- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f800b4e70 msgr2=0x7f7f80156d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f800b4e70 0x7f7f80156d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 -- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 msgr2=0x7f7f80149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 0x7f7f80149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f80156320 con 0x7f7f80070930 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0017070 con 0x7f56d4009ba0 2024-09-21T07:25:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56d000ce50 con 0x7f56d4009ba0 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 -- 172.21.15.105:0/1204429196 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c070d50 msgr2=0x7f0e9c071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 --2- 172.21.15.105:0/1204429196 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c070d50 0x7f0e9c071160 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005fd0 tx=0x7f0e9402b520 comp rx=0 tx=0).stop 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 -- 172.21.15.105:0/1204429196 shutdown_connections 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 --2- 172.21.15.105:0/1204429196 >> v2:172.21.15.105:3300/0 conn(0x7f0e9c073650 0x7f0e9c073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 --2- 172.21.15.105:0/1204429196 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c070d50 0x7f0e9c071160 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005fd0 tx=0x7f0e9402b520 comp rx=0 tx=0).stop 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 -- 172.21.15.105:0/1204429196 >> 172.21.15.105:0/1204429196 conn(0x7f0e9c0b1c80 msgr2=0x7f0e9c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 -- 172.21.15.105:0/1204429196 shutdown_connections 2024-09-21T07:25:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7fca417c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 0x7fca34186750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca417c6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 0x7fca340498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca417c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 0x7fca34022a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca417c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fca34052f60 con 0x7fca340e7360 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 0x7fca34022a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca3b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 0x7fca34022a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48588/0 (socket says 172.21.15.105:48588) 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca3b7fe640 1 -- 172.21.15.105:0/1454880138 learned_addr learned my addr 172.21.15.105:0/1454880138 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca417c6640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca34052de0 con 0x7fca34008850 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca417c6640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fca340530e0 con 0x7fca340e3510 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca3a7fc640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 0x7fca340498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7fca3affd640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 0x7fca34186750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7f840b178640 1 -- 172.21.15.105:0/1777588384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 msgr2=0x7f8404162a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.115+0000 7f840b178640 1 --2- 172.21.15.105:0/1777588384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404162a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8404072810 tx=0x7f83f402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f840b178640 1 -- 172.21.15.105:0/1777588384 shutdown_connections 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f840b178640 1 --2- 172.21.15.105:0/1777588384 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404162a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f840b178640 1 --2- 172.21.15.105:0/1777588384 >> v2:172.21.15.112:3300/0 conn(0x7f840415f940 0x7f84040710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f840b178640 1 -- 172.21.15.105:0/1777588384 >> 172.21.15.105:0/1777588384 conn(0x7f84040b1c80 msgr2=0x7f84040b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f56e2920640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d40d0f30 con 0x7f56d4009ba0 2024-09-21T07:25:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f56e2920640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d40d14f0 con 0x7f56d4009ba0 2024-09-21T07:25:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f84a01640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 0x7f7f80149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.113+0000 7f7f7ffff640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800cbf0 tx=0x7f7f7800a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.124 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0003830 con 0x7f56d4009ba0 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f56d001a070 con 0x7f56d4009ba0 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.116+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f56d001c780 con 0x7f56d4009ba0 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.114+0000 7f0ea1483640 1 -- 172.21.15.105:0/1204429196 wait complete. 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.117+0000 7f0ea1483640 1 Processor -- start 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7f0ea1483640 1 -- start start 2024-09-21T07:25:51.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7f840b178640 1 -- 172.21.15.105:0/1777588384 shutdown_connections 2024-09-21T07:25:51.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7f840b178640 1 -- 172.21.15.105:0/1777588384 wait complete. 2024-09-21T07:25:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.120+0000 7f0ea1483640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c073650 0x7f0e9c0c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea1483640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea1483640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e9c0af610 0x7f0e9c0c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea1483640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0b4640 con 0x7f0e9c0af610 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea1483640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c0b44c0 con 0x7f0e9c16b360 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea1483640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9c0b47c0 con 0x7f0e9c073650 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:57284/0 (socket says 172.21.15.105:57284) 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 -- 172.21.15.105:0/2991501160 learned_addr learned my addr 172.21.15.105:0/2991501160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0ea0c82640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e9c0af610 0x7f0e9c0c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 -- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c073650 msgr2=0x7f0e9c0c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c073650 0x7f0e9c0c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 -- 172.21.15.105:0/2991501160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e9c0af610 msgr2=0x7f0e9c0c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e9c0af610 0x7f0e9c0c68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f0e9b7fe640 1 -- 172.21.15.105:0/2991501160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c16a710 con 0x7f0e9c16b360 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7f0e9b7fe640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400afd0 tx=0x7f0e940329a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94037040 con 0x7f0e9c16b360 2024-09-21T07:25:51.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7f0ea1483640 1 -- 172.21.15.105:0/2991501160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c0c5090 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7f0ea1483640 1 -- 172.21.15.105:0/2991501160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c0c5650 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e9402f470 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9402fed0 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e94044020 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e940045b0 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.128+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e7c005290 con 0x7f0e9c16b360 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7fca3affd640 1 -- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 msgr2=0x7fca340498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7fca3affd640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 0x7fca340498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7fca3affd640 1 -- 172.21.15.105:0/1454880138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 msgr2=0x7fca34022a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7fca3affd640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 0x7fca34022a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.118+0000 7fca3affd640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3404c600 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.120+0000 7fca3affd640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 0x7fca34186750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca2c031000 tx=0x7fca2c033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c031b50 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca2c031de0 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7fca417c6640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca34186c90 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7fca417c6640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca34021c90 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.122+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c0332d0 con 0x7fca34008850 2024-09-21T07:25:51.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca2c033530 con 0x7fca34008850 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca2c03b770 con 0x7fca34008850 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca340e6e20 con 0x7fca34008850 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.127+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fca2c051020 con 0x7fca34008850 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9750005290 con 0x7f97680e7410 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f840b178640 1 Processor -- start 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7f840b178640 1 -- start start 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f840b178640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8404070270 0x7f8404150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f840b178640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f840b178640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f840415f940 0x7f8404142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f840b178640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f84040733a0 con 0x7f8404070680 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.123+0000 7f8409975640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48606/0 (socket says 172.21.15.105:48606) 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 -- 172.21.15.105:0/619309843 learned_addr learned my addr 172.21.15.105:0/619309843 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f840b178640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8404073220 con 0x7f840415f940 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f840b178640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8404073520 con 0x7f8404070270 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 -- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8404070270 msgr2=0x7f8404150eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8404070270 0x7f8404150eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 -- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f840415f940 msgr2=0x7f8404142b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f840415f940 0x7f8404142b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041500f0 con 0x7f8404070680 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f8409975640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c400 tx=0x7f83fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.121+0000 7fe2a27fc640 1 -- 172.21.15.105:0/4255119969 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fe2a80058e0 con 0x7fe2ac15ee80 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 msgr2=0x7fe2ac2063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 0x7fe2ac2063e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800b410 tx=0x7fe2a800b8d0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 shutdown_connections 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe2ac15f7d0 0x7fe2ac0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe2ac15ee80 0x7fe2ac2063e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 --2- 172.21.15.105:0/4255119969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2ac0ca970 0x7fe2ac1fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 >> 172.21.15.105:0/4255119969 conn(0x7fe2ac0ccb90 msgr2=0x7fe2ac1616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 shutdown_connections 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7fe297fff640 1 -- 172.21.15.105:0/4255119969 wait complete. 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.119+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56bc005290 con 0x7f56d4009ba0 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78016020 con 0x7f7f80070930 2024-09-21T07:25:51.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f7800acf0 con 0x7f7f80070930 2024-09-21T07:25:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.124+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f80158460 con 0x7f7f80070930 2024-09-21T07:25:51.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.120+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f439403f030 con 0x7f43a40d00b0 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.128+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011790 con 0x7f8404070680 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.129+0000 7f840b178640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84041503d0 con 0x7f8404070680 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.129+0000 7f840b178640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84040ce5c0 con 0x7f8404070680 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.131+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404151f00 con 0x7f8404070680 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.133+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f56d40ddcb0 con 0x7f56d4009ba0 2024-09-21T07:25:51.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.135+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f975c004dd0 con 0x7f97680e7410 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.135+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83fc017070 con 0x7f8404070680 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.136+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011c80 con 0x7f8404070680 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.136+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83fc006890 con 0x7f8404070680 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.136+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83fc01c460 con 0x7f8404070680 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.128+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f80158a20 con 0x7f7f80070930 2024-09-21T07:25:51.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.129+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78012680 con 0x7f7f80070930 2024-09-21T07:25:51.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.129+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f78019020 con 0x7f7f80070930 2024-09-21T07:25:51.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.137+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f7801a3b0 con 0x7f7f80070930 2024-09-21T07:25:51.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.137+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f60005290 con 0x7f7f80070930 2024-09-21T07:25:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.140+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0e9404b020 con 0x7f0e9c16b360 2024-09-21T07:25:51.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.141+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f83fc021020 con 0x7f8404070680 2024-09-21T07:25:51.146 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.147+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7f78020030 con 0x7f7f80070930 2024-09-21T07:25:51.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.157+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fc3fc005580 con 0x7fc4140340f0 2024-09-21T07:25:51.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.173+0000 7fc407fff640 1 -- 172.21.15.105:0/1808814801 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fc408004980 con 0x7fc4140340f0 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 msgr2=0x7fc4140ddc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 0x7fc4140ddc80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d6b0 tx=0x7fc40800db70 comp rx=0 tx=0).stop 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 shutdown_connections 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc414034a40 0x7fc4140de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc4140340f0 0x7fc4140ddc80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 --2- 172.21.15.105:0/1808814801 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc414005ea0 0x7fc4140ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 >> 172.21.15.105:0/1808814801 conn(0x7fc41401d9a0 msgr2=0x7fc414031e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 shutdown_connections 2024-09-21T07:25:51.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.183+0000 7fc405ffb640 1 -- 172.21.15.105:0/1808814801 wait complete. 2024-09-21T07:25:51.215 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.282+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f4384005580 con 0x7f43a40d00b0 2024-09-21T07:25:51.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.296+0000 7f439a7fc640 1 -- 172.21.15.105:0/33951962 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v12)=0 v12) ==== 125+0+0 (secure 0 0 0) 0x7f4394031770 con 0x7f43a40d00b0 2024-09-21T07:25:51.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.298+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 msgr2=0x7f43a40a99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.298+0000 7f438ffff640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 0x7f43a40a99c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4394031000 tx=0x7f4394033ba0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.297+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f4ea4005580 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.296+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fca3403aab0 con 0x7fca34008850 2024-09-21T07:25:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.303+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f3040005580 con 0x7f3060216090 2024-09-21T07:25:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.305+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f6680005580 con 0x7f66a006f9a0 2024-09-21T07:25:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.304+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 shutdown_connections 2024-09-21T07:25:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.304+0000 7f438ffff640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43a40d00b0 0x7f43a40a99c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.304+0000 7f438ffff640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43a40affd0 0x7f43a40a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.304+0000 7f438ffff640 1 --2- 172.21.15.105:0/33951962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f43a4070890 0x7f43a4155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.304+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 >> 172.21.15.105:0/33951962 conn(0x7f43a40d0910 msgr2=0x7f43a40b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.305+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 shutdown_connections 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.305+0000 7f438ffff640 1 -- 172.21.15.105:0/33951962 wait complete. 2024-09-21T07:25:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.306+0000 7f4ec0ff9640 1 -- 172.21.15.105:0/4255628283 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f4ebc02baa0 con 0x7f4ec40c6ef0 2024-09-21T07:25:51.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.311+0000 7fca337fe640 1 -- 172.21.15.105:0/1454880138 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v14)=0 v14) ==== 106+0+0 (secure 0 0 0) 0x7fca2c03f030 con 0x7fca34008850 2024-09-21T07:25:51.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.312+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 msgr2=0x7fca34186750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.312+0000 7fca317fa640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 0x7fca34186750 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca2c031000 tx=0x7fca2c033ba0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 shutdown_connections 2024-09-21T07:25:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fca340e7360 0x7fca34022a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fca340e3510 0x7fca340498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 --2- 172.21.15.105:0/1454880138 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca34008850 0x7fca34186750 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 >> 172.21.15.105:0/1454880138 conn(0x7fca3401eab0 msgr2=0x7fca34037cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 shutdown_connections 2024-09-21T07:25:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.316+0000 7fca317fa640 1 -- 172.21.15.105:0/1454880138 wait complete. 2024-09-21T07:25:51.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.317+0000 7f6687fff640 1 -- 172.21.15.105:0/3872424006 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f6690003c00 con 0x7f66a006f9a0 2024-09-21T07:25:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 msgr2=0x7f66a0074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f669000c370 tx=0x7f669000c770 comp rx=0 tx=0).stop 2024-09-21T07:25:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.319+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f56bc005580 con 0x7f56d4009ba0 2024-09-21T07:25:51.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 shutdown_connections 2024-09-21T07:25:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66a00cdce0 0x7f66a0216400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66a00c30e0 0x7f66a0074b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 --2- 172.21.15.105:0/3872424006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f66a006f9a0 0x7f66a0074600 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.320+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 >> 172.21.15.105:0/3872424006 conn(0x7f66a00ccb90 msgr2=0x7f66a0157a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.321+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 shutdown_connections 2024-09-21T07:25:51.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.321+0000 7f6685ffb640 1 -- 172.21.15.105:0/3872424006 wait complete. 2024-09-21T07:25:51.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.324+0000 7f305cff9640 1 -- 172.21.15.105:0/3211135238 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v16)=0 v16) ==== 119+0+0 (secure 0 0 0) 0x7f30602169e0 con 0x7f3060216090 2024-09-21T07:25:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 msgr2=0x7f4ec40c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f4eaa7fc640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009e50 tx=0x7f4ebc004250 comp rx=0 tx=0).stop 2024-09-21T07:25:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 msgr2=0x7f30602164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 0x7f30602164a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30600cda80 tx=0x7f30540079c0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 shutdown_connections 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3060216090 0x7f30602164a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3060215b30 0x7f306016ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 --2- 172.21.15.105:0/3211135238 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3060215720 0x7f306016a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 >> 172.21.15.105:0/3211135238 conn(0x7f30600cff40 msgr2=0x7f30600cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 shutdown_connections 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.325+0000 7f4eaa7fc640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ec40cf9d0 0x7f4ec41687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f4eaa7fc640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4ec40c6ef0 0x7f4ec40c4e80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 shutdown_connections 2024-09-21T07:25:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f4eaa7fc640 1 --2- 172.21.15.105:0/4255628283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ec40cf5c0 0x7f4ec41fde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 >> 172.21.15.105:0/4255628283 conn(0x7f4ec40cff40 msgr2=0x7f4ec408b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f30467fc640 1 -- 172.21.15.105:0/3211135238 wait complete. 2024-09-21T07:25:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 shutdown_connections 2024-09-21T07:25:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.326+0000 7f4eaa7fc640 1 -- 172.21.15.105:0/4255628283 wait complete. 2024-09-21T07:25:51.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.332+0000 7f56d8ff9640 1 -- 172.21.15.105:0/1998042106 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7f56d4006070 con 0x7f56d4009ba0 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.334+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 msgr2=0x7f56d4174e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.334+0000 7f56c67fc640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c7a0 tx=0x7f56d000cc60 comp rx=0 tx=0).stop 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.334+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 shutdown_connections 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f56d40dd360 0x7f56d40dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56d4009ba0 0x7f56d4174e90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 --2- 172.21.15.105:0/1998042106 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f56d4005c60 0x7f56d4021d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 >> 172.21.15.105:0/1998042106 conn(0x7f56d4051420 msgr2=0x7f56d40de460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 shutdown_connections 2024-09-21T07:25:51.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.335+0000 7f56c67fc640 1 -- 172.21.15.105:0/1998042106 wait complete. 2024-09-21T07:25:51.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.349+0000 7f7f86464640 1 -- 172.21.15.105:0/1133331144 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f7f60005850 con 0x7f7f80070930 2024-09-21T07:25:51.355 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.358 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.356+0000 7f7f7cff9640 1 -- 172.21.15.105:0/1133331144 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v18)=0 v18) ==== 108+0+0 (secure 0 0 0) 0x7f7f80156320 con 0x7f7f80070930 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.357+0000 7f7f667fc640 1 -- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 msgr2=0x7f7f801567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.357+0000 7f7f667fc640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800cbf0 tx=0x7f7f7800a430 comp rx=0 tx=0).stop 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 -- 172.21.15.105:0/1133331144 shutdown_connections 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f800b4e70 0x7f7f80156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f800cfd00 0x7f7f80149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 --2- 172.21.15.105:0/1133331144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7f80070930 0x7f7f801567e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 -- 172.21.15.105:0/1133331144 >> 172.21.15.105:0/1133331144 conn(0x7f7f8006cf80 msgr2=0x7f7f801591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 -- 172.21.15.105:0/1133331144 shutdown_connections 2024-09-21T07:25:51.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f7f667fc640 1 -- 172.21.15.105:0/1133331144 wait complete. 2024-09-21T07:25:51.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.358+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f0e7c005580 con 0x7f0e9c16b360 2024-09-21T07:25:51.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.361+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f63e00d07e0 con 0x7f63e0070930 2024-09-21T07:25:51.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.362+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f14140aebf0 con 0x7f1414162470 2024-09-21T07:25:51.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.364+0000 7f0e98ff9640 1 -- 172.21.15.105:0/2991501160 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f0e94004800 con 0x7f0e9c16b360 2024-09-21T07:25:51.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f9750005580 con 0x7f97680e7410 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 msgr2=0x7f0e9c0c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400afd0 tx=0x7f0e940329a0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 shutdown_connections 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e9c16b360 0x7f0e9c0c6390 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e9c0af610 0x7f0e9c0c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 --2- 172.21.15.105:0/2991501160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e9c073650 0x7f0e9c0c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.365+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 >> 172.21.15.105:0/2991501160 conn(0x7f0e9c0b1c80 msgr2=0x7f0e9c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.366+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 shutdown_connections 2024-09-21T07:25:51.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.366+0000 7f0e827fc640 1 -- 172.21.15.105:0/2991501160 wait complete. 2024-09-21T07:25:51.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.370+0000 7f976d7fa640 1 -- 172.21.15.105:0/415110746 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f975c003d30 con 0x7f97680e7410 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 msgr2=0x7f9768053520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 0x7f9768053520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9768052310 tx=0x7f975c0045e0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 shutdown_connections 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9768053a60 0x7f97680e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f97680e7410 0x7f9768053520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 --2- 172.21.15.105:0/415110746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9768052360 0x7f97680d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 >> 172.21.15.105:0/415110746 conn(0x7f976801d9a0 msgr2=0x7f976800ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 shutdown_connections 2024-09-21T07:25:51.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.372+0000 7f9776d24640 1 -- 172.21.15.105:0/415110746 wait complete. 2024-09-21T07:25:51.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.377+0000 7f140affd640 1 -- 172.21.15.105:0/3615064275 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v21)=0 v21) ==== 119+0+0 (secure 0 0 0) 0x7f1400004e10 con 0x7f1414162470 2024-09-21T07:25:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 msgr2=0x7f1414205930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 0x7f1414205930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1400031000 tx=0x7f1400003b90 comp rx=0 tx=0).stop 2024-09-21T07:25:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 shutdown_connections 2024-09-21T07:25:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1414162470 0x7f1414205930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14140ca970 0x7f14142053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 --2- 172.21.15.105:0/3615064275 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14140aefa0 0x7f14140b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 >> 172.21.15.105:0/3615064275 conn(0x7f14140ccb90 msgr2=0x7f14140b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 shutdown_connections 2024-09-21T07:25:51.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.378+0000 7f1408ff9640 1 -- 172.21.15.105:0/3615064275 wait complete. 2024-09-21T07:25:51.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.382+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f8404153b50 con 0x7f8404070680 2024-09-21T07:25:51.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.384+0000 7f63d6ffd640 1 -- 172.21.15.105:0/3273258172 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v22)=0 v22) ==== 111+0+0 (secure 0 0 0) 0x7f63d00193b0 con 0x7f63e0070930 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 msgr2=0x7f63e013f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b3e0 tx=0x7f63d000b8a0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 shutdown_connections 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63e00b4e70 0x7f63e014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63e00cfd00 0x7f63e0150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.385+0000 7f63e82bd640 1 --2- 172.21.15.105:0/3273258172 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f63e0070930 0x7f63e013f470 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.386+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 >> 172.21.15.105:0/3273258172 conn(0x7f63e006cf80 msgr2=0x7f63e00696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.386+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 shutdown_connections 2024-09-21T07:25:51.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.386+0000 7f63e82bd640 1 -- 172.21.15.105:0/3273258172 wait complete. 2024-09-21T07:25:51.395 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.395 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.395 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.395 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.396 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.395+0000 7f83faffd640 1 -- 172.21.15.105:0/619309843 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f83fc006b80 con 0x7f8404070680 2024-09-21T07:25:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 msgr2=0x7f8404151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00c400 tx=0x7f83fc00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 shutdown_connections 2024-09-21T07:25:51.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f840415f940 0x7f8404142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8404070680 0x7f8404151900 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 --2- 172.21.15.105:0/619309843 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8404070270 0x7f8404150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 >> 172.21.15.105:0/619309843 conn(0x7f84040b1c80 msgr2=0x7f8404162140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 shutdown_connections 2024-09-21T07:25:51.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.396+0000 7f83f8ff9640 1 -- 172.21.15.105:0/619309843 wait complete. 2024-09-21T07:25:51.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi105') 2024-09-21T07:25:51.409 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config dump 2024-09-21T07:25:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 -- 172.21.15.105:0/457018223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc15cd90 msgr2=0x7f32cc0af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 --2- 172.21.15.105:0/457018223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc15cd90 0x7f32cc0af6b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32b4009830 tx=0x7f32b402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 -- 172.21.15.105:0/457018223 shutdown_connections 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 --2- 172.21.15.105:0/457018223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc0b34c0 0x7f32cc15f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 --2- 172.21.15.105:0/457018223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc15cd90 0x7f32cc0af6b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 -- 172.21.15.105:0/457018223 >> 172.21.15.105:0/457018223 conn(0x7f32cc0b6610 msgr2=0x7f32cc0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 -- 172.21.15.105:0/457018223 shutdown_connections 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.558+0000 7f32d28ec640 1 -- 172.21.15.105:0/457018223 wait complete. 2024-09-21T07:25:51.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.560+0000 7f32d28ec640 1 Processor -- start 2024-09-21T07:25:51.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.560+0000 7f32d28ec640 1 -- start start 2024-09-21T07:25:51.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 0x7f32cc1f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc15f700 0x7f32cc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1560c0 con 0x7f32cc15cd90 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc155f40 con 0x7f32cc0b34c0 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32d28ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f32cc0b4d30 con 0x7f32cc15f700 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.561+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48612/0 (socket says 172.21.15.105:48612) 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 -- 172.21.15.105:0/3996230718 learned_addr learned my addr 172.21.15.105:0/3996230718 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cbfff640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 0x7f32cc1f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32d0e89640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc15f700 0x7f32cc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 -- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc15f700 msgr2=0x7f32cc156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc15f700 0x7f32cc156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 -- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 msgr2=0x7f32cc1f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 0x7f32cc1f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 -- 172.21.15.105:0/3996230718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1f9da0 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cbfff640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 0x7f32cc1f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32cb7fe640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9f0 tx=0x7f32bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc00cd80 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32bc004020 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc004450 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1f5cb0 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc0d02f0 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.562+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f32bc0053b0 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.563+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f32bc003ac0 con 0x7f32cc15cd90 2024-09-21T07:25:51.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.563+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc15f1c0 con 0x7f32cc15cd90 2024-09-21T07:25:51.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.565+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f32bc0093f0 con 0x7f32cc15cd90 2024-09-21T07:25:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.626+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f32cc1f9f70 con 0x7f32cc15cd90 2024-09-21T07:25:51.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.627+0000 7f32c8ff9640 1 -- 172.21.15.105:0/3996230718 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f32bc003cc0 con 0x7f32cc15cd90 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_client 20/20 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_mds 20/20 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:25:51.629 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_ms 2/2 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_debug_frag true 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_verify_scatter true 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced client_mount_timeout 600 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced debug_client 20/20 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced debug_ms 1/1 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:25:51.630 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:25:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.627+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 msgr2=0x7f32cc14ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.627+0000 7f32d28ec640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00e9f0 tx=0x7f32bc00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 shutdown_connections 2024-09-21T07:25:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f32cc15f700 0x7f32cc156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f32cc15cd90 0x7f32cc14ec10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 --2- 172.21.15.105:0/3996230718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32cc0b34c0 0x7f32cc1f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 >> 172.21.15.105:0/3996230718 conn(0x7f32cc0b6610 msgr2=0x7f32cc0b28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 shutdown_connections 2024-09-21T07:25:51.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.628+0000 7f32d28ec640 1 -- 172.21.15.105:0/3996230718 wait complete. 2024-09-21T07:25:51.640 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T07:25:51.640 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T07:25:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 -- 172.21.15.105:0/1116190670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b65d0 msgr2=0x7f637c0cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 --2- 172.21.15.105:0/1116190670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b65d0 0x7f637c0cc630 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f637c0b6e70 tx=0x7f637002a530 comp rx=0 tx=0).stop 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 -- 172.21.15.105:0/1116190670 shutdown_connections 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 --2- 172.21.15.105:0/1116190670 >> v2:172.21.15.112:3300/0 conn(0x7f637c0b0dc0 0x7f637c0b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 --2- 172.21.15.105:0/1116190670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b65d0 0x7f637c0cc630 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 -- 172.21.15.105:0/1116190670 >> 172.21.15.105:0/1116190670 conn(0x7f637c0b2d40 msgr2=0x7f637c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 -- 172.21.15.105:0/1116190670 shutdown_connections 2024-09-21T07:25:51.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.751+0000 7f6382cb6640 1 -- 172.21.15.105:0/1116190670 wait complete. 2024-09-21T07:25:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.752+0000 7f6382cb6640 1 Processor -- start 2024-09-21T07:25:51.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.753+0000 7f6382cb6640 1 -- start start 2024-09-21T07:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f637c0b65d0 0x7f637c1f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f637c0cf5f0 0x7f637c09db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f637c0b1cf0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f637c0b1b70 con 0x7f637c0b65d0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6382cb6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f637c0b1e70 con 0x7f637c0cf5f0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48620/0 (socket says 172.21.15.105:48620) 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 -- 172.21.15.105:0/3322592626 learned_addr learned my addr 172.21.15.105:0/3322592626 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6381253640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f637c0cf5f0 0x7f637c09db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f637bfff640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f637c0b65d0 0x7f637c1f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 -- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f637c0cf5f0 msgr2=0x7f637c09db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f637c0cf5f0 0x7f637c09db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 -- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f637c0b65d0 msgr2=0x7f637c1f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f637c0b65d0 0x7f637c1f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.754+0000 7f6380a52640 1 -- 172.21.15.105:0/3322592626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637c1f36f0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f6380a52640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6370009920 tx=0x7f6370033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370005c90 con 0x7f637c0b0dc0 2024-09-21T07:25:51.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6370031e50 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637c1f7410 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370005220 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637c1f79d0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63700047a0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.755+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63700333a0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.756+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637c0b11d0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.759+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f637003f030 con 0x7f637c0b0dc0 2024-09-21T07:25:51.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.824+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f637c1645e0 con 0x7f637c0b0dc0 2024-09-21T07:25:51.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.826+0000 7f63797fa640 1 -- 172.21.15.105:0/3322592626 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f6370031770 con 0x7f637c0b0dc0 2024-09-21T07:25:51.828 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.826+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 msgr2=0x7f637c0b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.826+0000 7f6382cb6640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6370009920 tx=0x7f6370033c20 comp rx=0 tx=0).stop 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 shutdown_connections 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f637c0b65d0 0x7f637c1f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f637c0cf5f0 0x7f637c09db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 --2- 172.21.15.105:0/3322592626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f637c0b0dc0 0x7f637c0b01c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 >> 172.21.15.105:0/3322592626 conn(0x7f637c0b2d40 msgr2=0x7f637c0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 shutdown_connections 2024-09-21T07:25:51.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:51.827+0000 7f6382cb6640 1 -- 172.21.15.105:0/3322592626 wait complete. 2024-09-21T07:25:51.839 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T07:25:51.839 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T07:25:51.840 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:51.840 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T07:25:51.864 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:51.864 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T07:25:51.928 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:51.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T07:25:51.993 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:25:51.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T07:25:52.059 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:52.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T07:25:52.084 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:52.084 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T07:25:52.149 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:52.149 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T07:25:52.215 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:25:52.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T07:25:52.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 066840df-9c4d-4c30-9039-9c24adc15c08 0 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.431+0000 7f89c847f640 1 -- 172.21.15.112:0/3403523220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 msgr2=0x7f89c015f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.431+0000 7f89c847f640 1 --2- 172.21.15.112:0/3403523220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c015f1c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009830 tx=0x7f89b402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.432+0000 7f89c847f640 1 -- 172.21.15.112:0/3403523220 shutdown_connections 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.432+0000 7f89c847f640 1 --2- 172.21.15.112:0/3403523220 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c015f1c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.432+0000 7f89c847f640 1 -- 172.21.15.112:0/3403523220 >> 172.21.15.112:0/3403523220 conn(0x7f89c00b6610 msgr2=0x7f89c00b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.432+0000 7f89c847f640 1 -- 172.21.15.112:0/3403523220 shutdown_connections 2024-09-21T07:25:52.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.432+0000 7f89c847f640 1 -- 172.21.15.112:0/3403523220 wait complete. 2024-09-21T07:25:52.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.433+0000 7f89c847f640 1 Processor -- start 2024-09-21T07:25:52.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.433+0000 7f89c847f640 1 -- start start 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c01f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0166d30 con 0x7f89c00b34c0 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0156040 con 0x7f89c015cd90 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f89c0166eb0 con 0x7f89c015f700 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c5a1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40038/0 (socket says 172.21.15.112:40038) 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c5a1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45508/0 (socket says 172.21.15.112:45508) 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 -- 172.21.15.112:0/937275976 learned_addr learned my addr 172.21.15.112:0/937275976 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c621b640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c01f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 -- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 msgr2=0x7f89c014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 -- 172.21.15.112:0/937275976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 msgr2=0x7f89c01f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c01f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 -- 172.21.15.112:0/937275976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c01f4430 con 0x7f89c015f700 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c5a1a640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c6a1c640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f89b800c760 tx=0x7f89b800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8003da0 con 0x7f89c015f700 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89b800ce10 con 0x7f89c015f700 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c01f4710 con 0x7f89c015f700 2024-09-21T07:25:52.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.435+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80052d0 con 0x7f89c015f700 2024-09-21T07:25:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.436+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c0151bb0 con 0x7f89c015f700 2024-09-21T07:25:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.436+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89b8005470 con 0x7f89c015f700 2024-09-21T07:25:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.436+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f89b801b8b0 con 0x7f89c015f700 2024-09-21T07:25:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.437+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c00cf960 con 0x7f89c015f700 2024-09-21T07:25:52.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.439+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f89c015fb10 con 0x7f89c015f700 2024-09-21T07:25:52.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.509+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "066840df-9c4d-4c30-9039-9c24adc15c08", "id": 0} v 0) -- 0x7f89c015f1c0 con 0x7f89c015f700 2024-09-21T07:25:52.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.513+0000 7f89b2ffd640 1 -- 172.21.15.112:0/937275976 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "066840df-9c4d-4c30-9039-9c24adc15c08", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f89b801bab0 con 0x7f89c015f700 2024-09-21T07:25:52.516 INFO:teuthology.orchestra.run.smithi112.stdout:0 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 msgr2=0x7f89c00d0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f89b800c760 tx=0x7f89b800cc20 comp rx=0 tx=0).stop 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 shutdown_connections 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89c015cd90 0x7f89c014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89c015f700 0x7f89c00d0370 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 --2- 172.21.15.112:0/937275976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89c00b34c0 0x7f89c01f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 >> 172.21.15.112:0/937275976 conn(0x7f89c00b6610 msgr2=0x7f89c00703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 shutdown_connections 2024-09-21T07:25:52.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.514+0000 7f89c847f640 1 -- 172.21.15.112:0/937275976 wait complete. 2024-09-21T07:25:52.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new e1787ece-29de-4aca-9dce-d142efa23f0d 1 2024-09-21T07:25:52.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 -- 172.21.15.112:0/1307012951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 msgr2=0x7f5b000af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 --2- 172.21.15.112:0/1307012951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b000af6b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5af4005fd0 tx=0x7f5af402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 -- 172.21.15.112:0/1307012951 shutdown_connections 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 --2- 172.21.15.112:0/1307012951 >> v2:172.21.15.105:3300/0 conn(0x7f5b0015f8d0 0x7f5b001554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 --2- 172.21.15.112:0/1307012951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b000af6b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 -- 172.21.15.112:0/1307012951 >> 172.21.15.112:0/1307012951 conn(0x7f5b000b6610 msgr2=0x7f5b000b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 -- 172.21.15.112:0/1307012951 shutdown_connections 2024-09-21T07:25:52.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.638+0000 7f5b07263640 1 -- 172.21.15.112:0/1307012951 wait complete. 2024-09-21T07:25:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.640+0000 7f5b07263640 1 Processor -- start 2024-09-21T07:25:52.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.640+0000 7f5b07263640 1 -- start start 2024-09-21T07:25:52.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b000b34c0 0x7f5b00146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b0015af60 0x7f5b00154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5b001560b0 con 0x7f5b0015af60 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b00155f30 con 0x7f5b000b34c0 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.641+0000 7f5b07263640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5b000b4d30 con 0x7f5b0015cd90 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40060/0 (socket says 172.21.15.112:40060) 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5b04fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b000b34c0 0x7f5b00146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 -- 172.21.15.112:0/3621989333 learned_addr learned my addr 172.21.15.112:0/3621989333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5b05800640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b0015af60 0x7f5b00154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 -- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b000b34c0 msgr2=0x7f5b00146890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b000b34c0 0x7f5b00146890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 -- 172.21.15.112:0/3621989333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b0015af60 msgr2=0x7f5b00154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b0015af60 0x7f5b00154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 -- 172.21.15.112:0/3621989333 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b0014b2d0 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afffff640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c3a0 tx=0x7f5aec00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec017070 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5aec0053c0 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b0014b570 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec003850 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.642+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b0014bb30 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.643+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5aec01a070 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.643+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f5aec011cf0 con 0x7f5b0015cd90 2024-09-21T07:25:52.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.644+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b000af6b0 con 0x7f5b0015cd90 2024-09-21T07:25:52.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.646+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5aec021070 con 0x7f5b0015cd90 2024-09-21T07:25:52.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.716+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e1787ece-29de-4aca-9dce-d142efa23f0d", "id": 1} v 0) -- 0x7f5b001e7dd0 con 0x7f5b0015cd90 2024-09-21T07:25:52.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.720+0000 7f5afd7fa640 1 -- 172.21.15.112:0/3621989333 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e1787ece-29de-4aca-9dce-d142efa23f0d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f5b000afbf0 con 0x7f5b0015cd90 2024-09-21T07:25:52.722 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 msgr2=0x7f5b001e4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c3a0 tx=0x7f5aec00c860 comp rx=0 tx=0).stop 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 shutdown_connections 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5b0015af60 0x7f5b00154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b0015cd90 0x7f5b001e4fb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 --2- 172.21.15.112:0/3621989333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b000b34c0 0x7f5b00146890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 >> 172.21.15.112:0/3621989333 conn(0x7f5b000b6610 msgr2=0x7f5b0014c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 shutdown_connections 2024-09-21T07:25:52.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.721+0000 7f5b07263640 1 -- 172.21.15.112:0/3621989333 wait complete. 2024-09-21T07:25:52.734 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new dff6f411-0843-43ba-a7a3-d855a7a96eac 2 2024-09-21T07:25:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 -- 172.21.15.112:0/2771418702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f680d01e0 msgr2=0x7f0f680b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/2771418702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f680d01e0 0x7f0f680b1a50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009830 tx=0x7f0f5402a640 comp rx=0 tx=0).stop 2024-09-21T07:25:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 -- 172.21.15.112:0/2771418702 shutdown_connections 2024-09-21T07:25:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/2771418702 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f680d01e0 0x7f0f680b1a50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 -- 172.21.15.112:0/2771418702 >> 172.21.15.112:0/2771418702 conn(0x7f0f680cedf0 msgr2=0x7f0f680cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 -- 172.21.15.112:0/2771418702 shutdown_connections 2024-09-21T07:25:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.845+0000 7f0f6edf1640 1 -- 172.21.15.112:0/2771418702 wait complete. 2024-09-21T07:25:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.847+0000 7f0f6edf1640 1 Processor -- start 2024-09-21T07:25:52.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.847+0000 7f0f6edf1640 1 -- start start 2024-09-21T07:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6809ba30 0x7f0f680c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:52.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68167e50 con 0x7f0f6809ba30 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f68167cd0 con 0x7f0f680d01e0 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6edf1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0f68167fd0 con 0x7f0f680d05f0 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6d38e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45514/0 (socket says 172.21.15.112:45514) 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6d38e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40064/0 (socket says 172.21.15.112:40064) 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f67fff640 1 -- 172.21.15.112:0/3967667601 learned_addr learned my addr 172.21.15.112:0/3967667601 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.848+0000 7f0f6cb8d640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6809ba30 0x7f0f680c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 -- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 msgr2=0x7f0f681574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 -- 172.21.15.112:0/3967667601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6809ba30 msgr2=0x7f0f680c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6809ba30 0x7f0f680c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 -- 172.21.15.112:0/3967667601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f680b6b90 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f6d38e640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f67fff640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009920 tx=0x7f0f54033df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f5402ae60 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f540332e0 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f68153eb0 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54005020 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f68154470 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0f54045020 con 0x7f0f680d01e0 2024-09-21T07:25:52.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.849+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0f54004de0 con 0x7f0f680d01e0 2024-09-21T07:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.850+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f680ce550 con 0x7f0f680d01e0 2024-09-21T07:25:52.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.852+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0f540047a0 con 0x7f0f680d01e0 2024-09-21T07:25:52.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.924+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dff6f411-0843-43ba-a7a3-d855a7a96eac", "id": 2} v 0) -- 0x7f0f680c9990 con 0x7f0f680d01e0 2024-09-21T07:25:52.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.927+0000 7f0f657fa640 1 -- 172.21.15.112:0/3967667601 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dff6f411-0843-43ba-a7a3-d855a7a96eac", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0f540035f0 con 0x7f0f680d01e0 2024-09-21T07:25:52.929 INFO:teuthology.orchestra.run.smithi112.stdout:2 2024-09-21T07:25:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.928+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 msgr2=0x7f0f6814bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.928+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009920 tx=0x7f0f54033df0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 shutdown_connections 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f680d01e0 0x7f0f6814bdf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f680d05f0 0x7f0f681574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 --2- 172.21.15.112:0/3967667601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6809ba30 0x7f0f680c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 >> 172.21.15.112:0/3967667601 conn(0x7f0f680cedf0 msgr2=0x7f0f680b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 shutdown_connections 2024-09-21T07:25:52.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:52.929+0000 7f0f6edf1640 1 -- 172.21.15.112:0/3967667601 wait complete. 2024-09-21T07:25:52.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new a9ea30f8-69d0-4b00-9c3a-c179c73733e1 3 2024-09-21T07:25:53.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 -- 172.21.15.112:0/2419558259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 msgr2=0x7f4e500af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 --2- 172.21.15.112:0/2419558259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e500af6b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4e40005fd0 tx=0x7f4e4002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 -- 172.21.15.112:0/2419558259 shutdown_connections 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 --2- 172.21.15.112:0/2419558259 >> v2:172.21.15.105:3300/0 conn(0x7f4e500b34c0 0x7f4e5015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 --2- 172.21.15.112:0/2419558259 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e500af6b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 -- 172.21.15.112:0/2419558259 >> 172.21.15.112:0/2419558259 conn(0x7f4e500b6610 msgr2=0x7f4e500b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 -- 172.21.15.112:0/2419558259 shutdown_connections 2024-09-21T07:25:53.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.050+0000 7f4e558f2640 1 -- 172.21.15.112:0/2419558259 wait complete. 2024-09-21T07:25:53.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.052+0000 7f4e558f2640 1 Processor -- start 2024-09-21T07:25:53.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.052+0000 7f4e558f2640 1 -- start start 2024-09-21T07:25:53.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.053+0000 7f4e558f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 0x7f4e5014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 0x7f4e500cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4e501560c0 con 0x7f4e5015f700 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e50155f40 con 0x7f4e500b34c0 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f4e500b4d30 con 0x7f4e5015cd90 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40088/0 (socket says 172.21.15.112:40088) 2024-09-21T07:25:53.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 -- 172.21.15.112:0/3279784823 learned_addr learned my addr 172.21.15.112:0/3279784823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4effd640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 0x7f4e5014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4f7fe640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 0x7f4e500cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 -- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 msgr2=0x7f4e5014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 0x7f4e5014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 -- 172.21.15.112:0/3279784823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 msgr2=0x7f4e500cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 0x7f4e500cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 -- 172.21.15.112:0/3279784823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e501e4aa0 con 0x7f4e5015cd90 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4effd640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 0x7f4e5014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:53.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4f7fe640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 0x7f4e500cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e4e7fc640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400e790 tx=0x7f4e4400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4400eea0 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e4400cc60 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e50154970 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44005280 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.054+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501e4d10 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.055+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e440047a0 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.055+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f4e44018430 con 0x7f4e5015cd90 2024-09-21T07:25:53.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.055+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5015f1c0 con 0x7f4e5015cd90 2024-09-21T07:25:53.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.057+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4e44004c20 con 0x7f4e5015cd90 2024-09-21T07:25:53.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.128+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a9ea30f8-69d0-4b00-9c3a-c179c73733e1", "id": 3} v 0) -- 0x7f4e5015d760 con 0x7f4e5015cd90 2024-09-21T07:25:53.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.132+0000 7f4e37fff640 1 -- 172.21.15.112:0/3279784823 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9ea30f8-69d0-4b00-9c3a-c179c73733e1", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f4e44004e00 con 0x7f4e5015cd90 2024-09-21T07:25:53.134 INFO:teuthology.orchestra.run.smithi112.stdout:3 2024-09-21T07:25:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 msgr2=0x7f4e50154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4e4400e790 tx=0x7f4e4400ec50 comp rx=0 tx=0).stop 2024-09-21T07:25:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 shutdown_connections 2024-09-21T07:25:53.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4e5015f700 0x7f4e500cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4e5015cd90 0x7f4e50154020 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 --2- 172.21.15.112:0/3279784823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e500b34c0 0x7f4e5014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 >> 172.21.15.112:0/3279784823 conn(0x7f4e500b6610 msgr2=0x7f4e500b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 shutdown_connections 2024-09-21T07:25:53.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.133+0000 7f4e558f2640 1 -- 172.21.15.112:0/3279784823 wait complete. 2024-09-21T07:25:53.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 3a5b5afe-ea5d-4d28-b083-763b9805cec2 4 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 -- 172.21.15.112:0/3300445753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00d0200 msgr2=0x7f6fc015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 --2- 172.21.15.112:0/3300445753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00d0200 0x7f6fc015d5d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8005fd0 tx=0x7f6fb802a980 comp rx=0 tx=0).stop 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 -- 172.21.15.112:0/3300445753 shutdown_connections 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 --2- 172.21.15.112:0/3300445753 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00d0200 0x7f6fc015d5d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 --2- 172.21.15.112:0/3300445753 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc00b0150 0x7f6fc0152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 -- 172.21.15.112:0/3300445753 >> 172.21.15.112:0/3300445753 conn(0x7f6fc006cf80 msgr2=0x7f6fc006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 -- 172.21.15.112:0/3300445753 shutdown_connections 2024-09-21T07:25:53.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.256+0000 7f6fc760e640 1 -- 172.21.15.112:0/3300445753 wait complete. 2024-09-21T07:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.259+0000 7f6fc760e640 1 Processor -- start 2024-09-21T07:25:53.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.259+0000 7f6fc760e640 1 -- start start 2024-09-21T07:25:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 0x7f6fc0142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fc00d0200 0x7f6fc013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc015e090 con 0x7f6fc00d0200 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc015df10 con 0x7f6fc00b0150 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc760e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc015e210 con 0x7f6fc015c630 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 0x7f6fc0142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc5bab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc5bab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40106/0 (socket says 172.21.15.112:40106) 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc53aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 0x7f6fc0142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45540/0 (socket says 172.21.15.112:45540) 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.261+0000 7f6fc5bab640 1 -- 172.21.15.112:0/2729742731 learned_addr learned my addr 172.21.15.112:0/2729742731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc4ba9640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fc00d0200 0x7f6fc013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 -- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 msgr2=0x7f6fc0142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 0x7f6fc0142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 -- 172.21.15.112:0/2729742731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fc00d0200 msgr2=0x7f6fc013ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fc00d0200 0x7f6fc013ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 -- 172.21.15.112:0/2729742731 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc013f180 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc5bab640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb8009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8034040 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb8004540 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8045480 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc0153420 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.262+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc01539e0 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.263+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fb8037020 con 0x7f6fc015c630 2024-09-21T07:25:53.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.263+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6fb80397a0 con 0x7f6fc015c630 2024-09-21T07:25:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.263+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc0153170 con 0x7f6fc015c630 2024-09-21T07:25:53.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.265+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6fb8039a70 con 0x7f6fc015c630 2024-09-21T07:25:53.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.336+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3a5b5afe-ea5d-4d28-b083-763b9805cec2", "id": 4} v 0) -- 0x7f6fc015e730 con 0x7f6fc015c630 2024-09-21T07:25:53.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.340+0000 7f6fb1ffb640 1 -- 172.21.15.112:0/2729742731 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3a5b5afe-ea5d-4d28-b083-763b9805cec2", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6fb8039c50 con 0x7f6fc015c630 2024-09-21T07:25:53.342 INFO:teuthology.orchestra.run.smithi112.stdout:4 2024-09-21T07:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 msgr2=0x7f6fc0159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8006010 tx=0x7f6fb8009360 comp rx=0 tx=0).stop 2024-09-21T07:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 shutdown_connections 2024-09-21T07:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6fc015c630 0x7f6fc0159a10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.343 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6fc00d0200 0x7f6fc013ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 --2- 172.21.15.112:0/2729742731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6fc00b0150 0x7f6fc0142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 >> 172.21.15.112:0/2729742731 conn(0x7f6fc006cf80 msgr2=0x7f6fc00b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 shutdown_connections 2024-09-21T07:25:53.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.341+0000 7f6fc760e640 1 -- 172.21.15.112:0/2729742731 wait complete. 2024-09-21T07:25:53.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 8a7765c0-85a7-4dde-805a-88680cc72d23 5 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 -- 172.21.15.112:0/916685975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a815f8d0 msgr2=0x7fd9a81554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/916685975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a815f8d0 0x7fd9a81554b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005fd0 tx=0x7fd9a002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 -- 172.21.15.112:0/916685975 shutdown_connections 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/916685975 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a815f8d0 0x7fd9a81554b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/916685975 >> v2:172.21.15.105:3300/0 conn(0x7fd9a815cd90 0x7fd9a80af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.464+0000 7fd9acaf9640 1 -- 172.21.15.112:0/916685975 >> 172.21.15.112:0/916685975 conn(0x7fd9a80b6610 msgr2=0x7fd9a80b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.465+0000 7fd9acaf9640 1 -- 172.21.15.112:0/916685975 shutdown_connections 2024-09-21T07:25:53.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.465+0000 7fd9acaf9640 1 -- 172.21.15.112:0/916685975 wait complete. 2024-09-21T07:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.468+0000 7fd9acaf9640 1 Processor -- start 2024-09-21T07:25:53.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.468+0000 7fd9acaf9640 1 -- start start 2024-09-21T07:25:53.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a815cd90 0x7fd9a814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 0x7fd9a814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81560c0 con 0x7fd9a815cd90 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8155f40 con 0x7fd9a814bd90 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9acaf9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a80b4d30 con 0x7fd9a80b34c0 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a6d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 0x7fd9a814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40136/0 (socket says 172.21.15.112:40136) 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 -- 172.21.15.112:0/1484115815 learned_addr learned my addr 172.21.15.112:0/1484115815 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a5d9b640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a815cd90 0x7fd9a814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 -- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 msgr2=0x7fd9a814cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 0x7fd9a814cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 -- 172.21.15.112:0/1484115815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a815cd90 msgr2=0x7fd9a814b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a815cd90 0x7fd9a814b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a659c640 1 -- 172.21.15.112:0/1484115815 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a814bb80 con 0x7fd9a80b34c0 2024-09-21T07:25:53.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.470+0000 7fd9a6d9d640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 0x7fd9a814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd9a659c640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e7c0 tx=0x7fd99800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99800ccc0 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd998010430 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a80d0980 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998010860 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a814fc00 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd998010a00 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.471+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fd998018c40 con 0x7fd9a80b34c0 2024-09-21T07:25:53.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.472+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a80af6b0 con 0x7fd9a80b34c0 2024-09-21T07:25:53.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.474+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd998018420 con 0x7fd9a80b34c0 2024-09-21T07:25:53.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.544+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8a7765c0-85a7-4dde-805a-88680cc72d23", "id": 5} v 0) -- 0x7fd9a80cf090 con 0x7fd9a80b34c0 2024-09-21T07:25:53.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.548+0000 7fd99effd640 1 -- 172.21.15.112:0/1484115815 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8a7765c0-85a7-4dde-805a-88680cc72d23", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fd998018600 con 0x7fd9a80b34c0 2024-09-21T07:25:53.549 INFO:teuthology.orchestra.run.smithi112.stdout:5 2024-09-21T07:25:53.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 msgr2=0x7fd9a80cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e7c0 tx=0x7fd99800ec80 comp rx=0 tx=0).stop 2024-09-21T07:25:53.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 shutdown_connections 2024-09-21T07:25:53.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a814bd90 0x7fd9a814cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd9a815cd90 0x7fd9a814b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 --2- 172.21.15.112:0/1484115815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9a80b34c0 0x7fd9a80cf470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 >> 172.21.15.112:0/1484115815 conn(0x7fd9a80b6610 msgr2=0x7fd9a81e72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 shutdown_connections 2024-09-21T07:25:53.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.549+0000 7fd9acaf9640 1 -- 172.21.15.112:0/1484115815 wait complete. 2024-09-21T07:25:53.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new efa5ac76-72bc-4495-9353-b42e9421b46b 6 2024-09-21T07:25:53.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 -- 172.21.15.112:0/1738585767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015cd90 msgr2=0x7f1a200af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 --2- 172.21.15.112:0/1738585767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015cd90 0x7f1a200af6b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a20073790 tx=0x7f1a1002a560 comp rx=0 tx=0).stop 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 -- 172.21.15.112:0/1738585767 shutdown_connections 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 --2- 172.21.15.112:0/1738585767 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a2015f700 0x7f1a201554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 --2- 172.21.15.112:0/1738585767 >> v2:172.21.15.105:3300/0 conn(0x7f1a200b34c0 0x7f1a2015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 --2- 172.21.15.112:0/1738585767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015cd90 0x7f1a200af6b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 -- 172.21.15.112:0/1738585767 >> 172.21.15.112:0/1738585767 conn(0x7f1a200b6610 msgr2=0x7f1a200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 -- 172.21.15.112:0/1738585767 shutdown_connections 2024-09-21T07:25:53.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.672+0000 7f1a2678e640 1 -- 172.21.15.112:0/1738585767 wait complete. 2024-09-21T07:25:53.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.674+0000 7f1a2678e640 1 Processor -- start 2024-09-21T07:25:53.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.674+0000 7f1a2678e640 1 -- start start 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 0x7f1a201463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 0x7f1a201f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1a20155e90 con 0x7f1a2015cd90 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a20155d10 con 0x7f1a2015f700 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a2678e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1a20156010 con 0x7f1a200b34c0 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a24d2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 0x7f1a201f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a1ffff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40152/0 (socket says 172.21.15.112:40152) 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.675+0000 7f1a1ffff640 1 -- 172.21.15.112:0/175408073 learned_addr learned my addr 172.21.15.112:0/175408073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 0x7f1a201463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 -- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 msgr2=0x7f1a200ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 -- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 msgr2=0x7f1a201f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 0x7f1a201f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 -- 172.21.15.112:0/175408073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a200b0da0 con 0x7f1a2015cd90 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1ffff640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a24d2b640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 0x7f1a201f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1f7fe640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 0x7f1a201463c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800e680 tx=0x7f1a0800ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:53.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800c9e0 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a0800cc70 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a200b1080 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a08010430 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a20147250 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a080105d0 con 0x7f1a2015cd90 2024-09-21T07:25:53.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.676+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f1a08018ab0 con 0x7f1a2015cd90 2024-09-21T07:25:53.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.678+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a200af6b0 con 0x7f1a2015cd90 2024-09-21T07:25:53.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.680+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1a08018d50 con 0x7f1a2015cd90 2024-09-21T07:25:53.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.750+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "efa5ac76-72bc-4495-9353-b42e9421b46b", "id": 6} v 0) -- 0x7f1a2014dce0 con 0x7f1a2015cd90 2024-09-21T07:25:53.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.752+0000 7f1a1cff9640 1 -- 172.21.15.112:0/175408073 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "efa5ac76-72bc-4495-9353-b42e9421b46b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f1a08018420 con 0x7f1a2015cd90 2024-09-21T07:25:53.754 INFO:teuthology.orchestra.run.smithi112.stdout:6 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 msgr2=0x7f1a201463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 0x7f1a201463c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800e680 tx=0x7f1a0800ea80 comp rx=0 tx=0).stop 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 shutdown_connections 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a2015f700 0x7f1a201f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1a2015cd90 0x7f1a201463c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 --2- 172.21.15.112:0/175408073 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1a200b34c0 0x7f1a200ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 >> 172.21.15.112:0/175408073 conn(0x7f1a200b6610 msgr2=0x7f1a2015be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 shutdown_connections 2024-09-21T07:25:53.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.753+0000 7f1a2678e640 1 -- 172.21.15.112:0/175408073 wait complete. 2024-09-21T07:25:53.766 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 52bd24cd-a720-4853-97f8-c5ceb4013988 7 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 -- 172.21.15.112:0/3344358104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde8015cd90 msgr2=0x7fde800af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 --2- 172.21.15.112:0/3344358104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde8015cd90 0x7fde800af6b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fde70005fd0 tx=0x7fde7002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 -- 172.21.15.112:0/3344358104 shutdown_connections 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 --2- 172.21.15.112:0/3344358104 >> v2:172.21.15.105:3300/0 conn(0x7fde8015f8d0 0x7fde801554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 --2- 172.21.15.112:0/3344358104 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde8015cd90 0x7fde800af6b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 -- 172.21.15.112:0/3344358104 >> 172.21.15.112:0/3344358104 conn(0x7fde800b6610 msgr2=0x7fde800b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 -- 172.21.15.112:0/3344358104 shutdown_connections 2024-09-21T07:25:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.877+0000 7fde88607640 1 -- 172.21.15.112:0/3344358104 wait complete. 2024-09-21T07:25:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.879+0000 7fde88607640 1 Processor -- start 2024-09-21T07:25:53.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.879+0000 7fde88607640 1 -- start start 2024-09-21T07:25:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde8015cd90 0x7fde800ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 0x7fde800ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:53.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fde801560b0 con 0x7fde8015cd90 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde80155f30 con 0x7fde800b34c0 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde88607640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fde800b4d30 con 0x7fde800d1960 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde86ba4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 0x7fde800ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45578/0 (socket says 172.21.15.112:45578) 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 -- 172.21.15.112:0/3572656590 learned_addr learned my addr 172.21.15.112:0/3572656590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde86ba4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 0x7fde800ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:40172/0 (socket says 172.21.15.112:40172) 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde85ba2640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde8015cd90 0x7fde800ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 -- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 msgr2=0x7fde800ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 0x7fde800ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 -- 172.21.15.112:0/3572656590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde8015cd90 msgr2=0x7fde800ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde8015cd90 0x7fde800ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.880+0000 7fde863a3640 1 -- 172.21.15.112:0/3572656590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde800cf310 con 0x7fde800b34c0 2024-09-21T07:25:53.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde863a3640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde70000c00 tx=0x7fde70005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7003e030 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde70003a70 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde70035040 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde800cf5f0 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8014f050 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde70041030 con 0x7fde800b34c0 2024-09-21T07:25:53.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.881+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fde70031400 con 0x7fde800b34c0 2024-09-21T07:25:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.882+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde800cf830 con 0x7fde800b34c0 2024-09-21T07:25:53.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.884+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fde800cf830 con 0x7fde800b34c0 2024-09-21T07:25:53.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.953+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "52bd24cd-a720-4853-97f8-c5ceb4013988", "id": 7} v 0) -- 0x7fde8015c590 con 0x7fde800b34c0 2024-09-21T07:25:53.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.957+0000 7fde76ffd640 1 -- 172.21.15.112:0/3572656590 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "52bd24cd-a720-4853-97f8-c5ceb4013988", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fde8015c590 con 0x7fde800b34c0 2024-09-21T07:25:53.959 INFO:teuthology.orchestra.run.smithi112.stdout:7 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 msgr2=0x7fde800d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fde70000c00 tx=0x7fde70005770 comp rx=0 tx=0).stop 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 shutdown_connections 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fde800d1960 0x7fde800ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fde8015cd90 0x7fde800ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 --2- 172.21.15.112:0/3572656590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde800b34c0 0x7fde800d1420 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 >> 172.21.15.112:0/3572656590 conn(0x7fde800b6610 msgr2=0x7fde801e5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 shutdown_connections 2024-09-21T07:25:53.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:25:53.959+0000 7fde88607640 1 -- 172.21.15.112:0/3572656590 wait complete. 2024-09-21T07:25:53.975 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T07:25:53.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T07:25:53.977 INFO:tasks.ceph.osd.0:Started 2024-09-21T07:25:53.977 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T07:25:53.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T07:25:53.979 INFO:tasks.ceph.osd.1:Started 2024-09-21T07:25:53.979 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T07:25:53.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T07:25:53.981 INFO:tasks.ceph.osd.2:Started 2024-09-21T07:25:53.982 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T07:25:53.982 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T07:25:53.984 INFO:tasks.ceph.osd.3:Started 2024-09-21T07:25:53.984 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T07:25:53.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T07:25:53.986 INFO:tasks.ceph.osd.4:Started 2024-09-21T07:25:53.986 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T07:25:53.986 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T07:25:53.988 INFO:tasks.ceph.osd.5:Started 2024-09-21T07:25:53.989 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T07:25:53.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T07:25:53.990 INFO:tasks.ceph.osd.6:Started 2024-09-21T07:25:53.991 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T07:25:53.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T07:25:53.993 INFO:tasks.ceph.osd.7:Started 2024-09-21T07:25:53.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 -- 172.21.15.105:0/1890001975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 msgr2=0x7f5f0c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 --2- 172.21.15.105:0/1890001975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c0cf3c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c06a0f0 tx=0x7f5f0402aac0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 -- 172.21.15.105:0/1890001975 shutdown_connections 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 --2- 172.21.15.105:0/1890001975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c0cf3c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 --2- 172.21.15.105:0/1890001975 >> v2:172.21.15.112:3300/0 conn(0x7f5f0c0b6690 0x7f5f0c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.158+0000 7f5f113c3640 1 -- 172.21.15.105:0/1890001975 >> 172.21.15.105:0/1890001975 conn(0x7f5f0c09d830 msgr2=0x7f5f0c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.159+0000 7f5f113c3640 1 -- 172.21.15.105:0/1890001975 shutdown_connections 2024-09-21T07:25:54.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.159+0000 7f5f113c3640 1 -- 172.21.15.105:0/1890001975 wait complete. 2024-09-21T07:25:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.160+0000 7f5f113c3640 1 Processor -- start 2024-09-21T07:25:54.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.160+0000 7f5f113c3640 1 -- start start 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f0c0d5080 0x7f5f0c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f0c156da0 0x7f5f0c0b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0cfe30 con 0x7f5f0c06f300 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c0cfcb0 con 0x7f5f0c0d5080 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0c0cffb0 con 0x7f5f0c156da0 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48654/0 (socket says 172.21.15.105:48654) 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 -- 172.21.15.105:0/3775090488 learned_addr learned my addr 172.21.15.105:0/3775090488 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:54.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0a7fc640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f0c0d5080 0x7f5f0c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0b7fe640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f0c156da0 0x7f5f0c0b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 -- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f0c156da0 msgr2=0x7f5f0c0b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f0c156da0 0x7f5f0c0b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 -- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f0c0d5080 msgr2=0x7f5f0c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f0c0d5080 0x7f5f0c156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 -- 172.21.15.105:0/3775090488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c0ca5a0 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f0affd640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c390 tx=0x7f5efc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc017070 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5efc0035f0 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc003a40 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.162+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c157be0 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.163+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c1581a0 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.163+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5efc01a070 con 0x7f5f0c06f300 2024-09-21T07:25:54.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.163+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5efc012ad0 con 0x7f5f0c06f300 2024-09-21T07:25:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.163+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c0cee80 con 0x7f5f0c06f300 2024-09-21T07:25:54.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.166+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5efc021020 con 0x7f5f0c06f300 2024-09-21T07:25:54.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.226+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f0c0cafb0 con 0x7f5f0c06f300 2024-09-21T07:25:54.228 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.226+0000 7f5f037fe640 1 -- 172.21.15.105:0/3775090488 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5efc003cc0 con 0x7f5f0c06f300 2024-09-21T07:25:54.229 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:25:54.229 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:25:53.955886+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 msgr2=0x7f5f0c157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c390 tx=0x7f5efc00c850 comp rx=0 tx=0).stop 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 shutdown_connections 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f0c0d5080 0x7f5f0c156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f0c156da0 0x7f5f0c0b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.227+0000 7f5f113c3640 1 --2- 172.21.15.105:0/3775090488 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f0c06f300 0x7f5f0c157270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.228+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 >> 172.21.15.105:0/3775090488 conn(0x7f5f0c09d830 msgr2=0x7f5f0c0b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.228+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 shutdown_connections 2024-09-21T07:25:54.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:25:54.228+0000 7f5f113c3640 1 -- 172.21.15.105:0/3775090488 wait complete. 2024-09-21T07:25:54.240 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T07:25:54.241 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T07:25:55.046 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.148+0000 7f18078cc640 1 -- 172.21.15.105:0/720448276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18000b5f00 msgr2=0x7f18000b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.148+0000 7f18078cc640 1 --2- 172.21.15.105:0/720448276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18000b5f00 0x7f18000b6310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009900 tx=0x7f17f002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.148+0000 7f18078cc640 1 -- 172.21.15.105:0/720448276 shutdown_connections 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.148+0000 7f18078cc640 1 --2- 172.21.15.105:0/720448276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 0x7f180015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.148+0000 7f18078cc640 1 --2- 172.21.15.105:0/720448276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18000b5f00 0x7f18000b6310 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.149+0000 7f18078cc640 1 -- 172.21.15.105:0/720448276 >> 172.21.15.105:0/720448276 conn(0x7f180006cda0 msgr2=0x7f180006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:55.151 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.149+0000 7f18078cc640 1 -- 172.21.15.105:0/720448276 shutdown_connections 2024-09-21T07:25:55.152 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.149+0000 7f18078cc640 1 -- 172.21.15.105:0/720448276 wait complete. 2024-09-21T07:25:55.152 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.150+0000 7f18078cc640 1 Processor -- start 2024-09-21T07:25:55.152 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.150+0000 7f18078cc640 1 -- start start 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b5f00 0x7f18001414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 0x7f18000b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f180015eda0 con 0x7f180015fc50 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f180015ec20 con 0x7f18000b5f00 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f180015ef20 con 0x7f180015e0c0 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:48684/0 (socket says 172.21.15.105:48684) 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 -- 172.21.15.105:0/2530174450 learned_addr learned my addr 172.21.15.105:0/2530174450 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805668640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b5f00 0x7f18001414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:55.155 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1804e67640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 0x7f18000b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 -- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 msgr2=0x7f18000b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 0x7f18000b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 -- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b5f00 msgr2=0x7f18001414a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b5f00 0x7f18001414a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 -- 172.21.15.105:0/2530174450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18001f9c80 con 0x7f180015fc50 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f1805e69640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ea80 tx=0x7f17f800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800cde0 con 0x7f180015fc50 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17f8004b20 con 0x7f180015fc50 2024-09-21T07:25:55.156 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001f59c0 con 0x7f180015fc50 2024-09-21T07:25:55.157 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.152+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8005020 con 0x7f180015fc50 2024-09-21T07:25:55.157 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.153+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001f5f80 con 0x7f180015fc50 2024-09-21T07:25:55.157 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.154+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f17f8005290 con 0x7f180015fc50 2024-09-21T07:25:55.157 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.154+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1800158360 con 0x7f180015fc50 2024-09-21T07:25:55.158 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.154+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f17f8052f70 con 0x7f180015fc50 2024-09-21T07:25:55.158 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.156+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f17f801b160 con 0x7f180015fc50 2024-09-21T07:25:55.219 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.217+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18000b6310 con 0x7f180015fc50 2024-09-21T07:25:55.220 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.218+0000 7f17f5ffb640 1 -- 172.21.15.105:0/2530174450 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f17f8021cf0 con 0x7f180015fc50 2024-09-21T07:25:55.220 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-21T07:25:55.220 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:25:53.955886+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 msgr2=0x7f18001e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17f800ea80 tx=0x7f17f800ef40 comp rx=0 tx=0).stop 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 shutdown_connections 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f180015fc50 0x7f18001e9350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f180015e0c0 0x7f18000b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 --2- 172.21.15.105:0/2530174450 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18000b5f00 0x7f18001414a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 >> 172.21.15.105:0/2530174450 conn(0x7f180006cda0 msgr2=0x7f1800155e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:25:55.221 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 shutdown_connections 2024-09-21T07:25:55.222 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:25:55.219+0000 7f18078cc640 1 -- 172.21.15.105:0/2530174450 wait complete. 2024-09-21T07:25:55.222 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-09-21T07:25:55.220+0000 7f7413a0d640 -1 Falling back to public interface 2024-09-21T07:25:55.232 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T07:25:55.492 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-09-21T07:25:55.491+0000 7ff34a40d640 -1 Falling back to public interface 2024-09-21T07:25:55.703 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-09-21T07:25:55.702+0000 7f63a922e640 -1 Falling back to public interface 2024-09-21T07:25:55.707 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-09-21T07:25:55.706+0000 7f7b2ae87640 -1 Falling back to public interface 2024-09-21T07:25:55.727 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-21T07:25:55.726+0000 7f35d10e8640 -1 Falling back to public interface 2024-09-21T07:25:55.731 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-21T07:25:55.730+0000 7fe04a014640 -1 Falling back to public interface 2024-09-21T07:25:55.754 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-21T07:25:55.752+0000 7f8d5fa20640 -1 Falling back to public interface 2024-09-21T07:25:55.773 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:25:55.771+0000 7f219ff5e640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T07:25:55.776 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:25:55.774+0000 7f219ff5e640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T07:25:55.939 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-09-21T07:25:55.938+0000 7f0d6140d640 -1 Falling back to public interface 2024-09-21T07:25:59.052 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-09-21T07:25:59.050+0000 7f7413a0d640 -1 osd.3 0 log_to_monitors true 2024-09-21T07:25:59.310 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-09-21T07:25:59.309+0000 7ff34a40d640 -1 osd.7 0 log_to_monitors true 2024-09-21T07:25:59.311 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-09-21T07:25:59.309+0000 7f7b2ae87640 -1 osd.6 0 log_to_monitors true 2024-09-21T07:25:59.341 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-21T07:25:59.339+0000 7fe04a014640 -1 osd.1 0 log_to_monitors true 2024-09-21T07:25:59.492 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-09-21T07:25:59.490+0000 7f63a922e640 -1 osd.4 0 log_to_monitors true 2024-09-21T07:25:59.543 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-21T07:25:59.541+0000 7f35d10e8640 -1 osd.2 0 log_to_monitors true 2024-09-21T07:25:59.861 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-21T07:25:59.859+0000 7f8d5fa20640 -1 osd.0 0 log_to_monitors true 2024-09-21T07:26:00.194 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-09-21T07:26:00.192+0000 7f0d6140d640 -1 osd.5 0 log_to_monitors true 2024-09-21T07:26:00.937 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-09-21T07:26:00.935+0000 7f740f44a640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T07:26:00.951 INFO:tasks.ceph.osd.3.smithi105.stderr:2024-09-21T07:26:00.950+0000 7f7407240640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:02.038 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:02.160 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 -- 172.21.15.105:0/3627237263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f54004800 msgr2=0x7f0f54004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:02.160 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 --2- 172.21.15.105:0/3627237263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f54004800 0x7f0f54004c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0f60001690 tx=0x7f0f54005830 comp rx=0 tx=0).stop 2024-09-21T07:26:02.160 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 -- 172.21.15.105:0/3627237263 shutdown_connections 2024-09-21T07:26:02.160 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 --2- 172.21.15.105:0/3627237263 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f54004800 0x7f0f54004c10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.160 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 -- 172.21.15.105:0/3627237263 >> 172.21.15.105:0/3627237263 conn(0x7f0f6c0cee40 msgr2=0x7f0f6c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:02.161 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 -- 172.21.15.105:0/3627237263 shutdown_connections 2024-09-21T07:26:02.161 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.157+0000 7f0f70a38640 1 -- 172.21.15.105:0/3627237263 wait complete. 2024-09-21T07:26:02.161 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.159+0000 7f0f70a38640 1 Processor -- start 2024-09-21T07:26:02.161 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.159+0000 7f0f70a38640 1 -- start start 2024-09-21T07:26:02.163 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f54004800 0x7f0f6c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f6c15b5a0 0x7f0f6c15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d0500 con 0x7f0f6c0c8420 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c0d0380 con 0x7f0f54004800 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6c0ce710 con 0x7f0f6c15b5a0 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50666/0 (socket says 172.21.15.105:50666) 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 -- 172.21.15.105:0/1020670922 learned_addr learned my addr 172.21.15.105:0/1020670922 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f69d9b640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f6c15b5a0 0x7f0f6c15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6a59c640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f54004800 0x7f0f6c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 -- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f6c15b5a0 msgr2=0x7f0f6c15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f6c15b5a0 0x7f0f6c15c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 -- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f54004800 msgr2=0x7f0f6c153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:02.164 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f54004800 0x7f0f6c153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 -- 172.21.15.105:0/1020670922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6c15bb40 con 0x7f0f6c0c8420 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f6ad9d640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f640013d0 tx=0x7f0f64006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400fa20 con 0x7f0f6c0c8420 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f64018070 con 0x7f0f6c0c8420 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6c152980 con 0x7f0f6c0c8420 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6400fe80 con 0x7f0f6c0c8420 2024-09-21T07:26:02.165 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.161+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f6c152f40 con 0x7f0f6c0c8420 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.162+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f6c071c90 con 0x7f0f6c0c8420 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.163+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f0f6401d460 con 0x7f0f6c0c8420 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.163+0000 7f0f5affd640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 0x7f0f4c07b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.164+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2495+0+0 (secure 0 0 0) 0x7f0f6409a5f0 con 0x7f0f6c0c8420 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.164+0000 7f0f6a59c640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 0x7f0f4c07b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:02.166 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.164+0000 7f0f6a59c640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 0x7f0f4c07b040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009cc0 tx=0x7f0f5c005f30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:02.170 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.169+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f6c071c90 con 0x7f0f6c0c8420 2024-09-21T07:26:02.467 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.465+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f6c0b1a90 con 0x7f0f6c0c8420 2024-09-21T07:26:02.468 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.466+0000 7f0f5affd640 1 -- 172.21.15.105:0/1020670922 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7f0f64063720 con 0x7f0f6c0c8420 2024-09-21T07:26:02.468 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-21T07:26:02.468 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":14,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:02.109039+0000","last_up_change":"2024-09-21T07:26:01.107594+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:02.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.469+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 msgr2=0x7f0f4c07b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:02.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.469+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 0x7f0f4c07b040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c009cc0 tx=0x7f0f5c005f30 comp rx=0 tx=0).stop 2024-09-21T07:26:02.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.469+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 msgr2=0x7f0f6c0b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:02.471 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.469+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0f640013d0 tx=0x7f0f64006660 comp rx=0 tx=0).stop 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 shutdown_connections 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0f4c078b30 0x7f0f4c07b040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0f6c0c8420 0x7f0f6c0b02b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f6c15b5a0 0x7f0f6c15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 --2- 172.21.15.105:0/1020670922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f54004800 0x7f0f6c153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 >> 172.21.15.105:0/1020670922 conn(0x7f0f6c0cee40 msgr2=0x7f0f6c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 shutdown_connections 2024-09-21T07:26:02.472 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:02.470+0000 7f0f70a38640 1 -- 172.21.15.105:0/1020670922 wait complete. 2024-09-21T07:26:02.485 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T07:26:02.879 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-21T07:26:02.878+0000 7fe04589e640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T07:26:02.890 INFO:tasks.ceph.osd.1.smithi105.stderr:2024-09-21T07:26:02.889+0000 7fe03d694640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:02.897 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-21T07:26:02.896+0000 7f8d5c1c3640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T07:26:02.904 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-21T07:26:02.902+0000 7f35cd076640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T07:26:02.913 INFO:tasks.ceph.osd.0.smithi105.stderr:2024-09-21T07:26:02.912+0000 7f8d537a4640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:02.917 INFO:tasks.ceph.osd.2.smithi105.stderr:2024-09-21T07:26:02.916+0000 7f35c4e6c640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:03.011 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-09-21T07:26:03.009+0000 7f63a4784640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T07:26:03.028 INFO:tasks.ceph.osd.4.smithi112.stderr:2024-09-21T07:26:03.027+0000 7f639c57a640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:03.054 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-09-21T07:26:03.052+0000 7ff345d1d640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T07:26:03.065 INFO:tasks.ceph.osd.7.smithi112.stderr:2024-09-21T07:26:03.063+0000 7ff33db13640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:03.087 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-09-21T07:26:03.085+0000 7f7b2762a640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T07:26:03.097 INFO:tasks.ceph.osd.6.smithi112.stderr:2024-09-21T07:26:03.096+0000 7f7b1ec0b640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:03.307 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-09-21T07:26:03.305+0000 7f0d5ce4a640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T07:26:03.317 INFO:tasks.ceph.osd.5.smithi112.stderr:2024-09-21T07:26:03.316+0000 7f0d54c40640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:26:09.291 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 -- 172.21.15.105:0/2038704945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200744f0 msgr2=0x7f5a200ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 --2- 172.21.15.105:0/2038704945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200744f0 0x7f5a200ce800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a10005fd0 tx=0x7f5a1002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 -- 172.21.15.105:0/2038704945 shutdown_connections 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 --2- 172.21.15.105:0/2038704945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200744f0 0x7f5a200ce800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 -- 172.21.15.105:0/2038704945 >> 172.21.15.105:0/2038704945 conn(0x7f5a200cdce0 msgr2=0x7f5a200ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 -- 172.21.15.105:0/2038704945 shutdown_connections 2024-09-21T07:26:09.407 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.405+0000 7f5a279e4640 1 -- 172.21.15.105:0/2038704945 wait complete. 2024-09-21T07:26:09.408 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.407+0000 7f5a279e4640 1 Processor -- start 2024-09-21T07:26:09.408 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.407+0000 7f5a279e4640 1 -- start start 2024-09-21T07:26:09.411 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2006ffd0 0x7f5a200b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.411 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a200744f0 0x7f5a200b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.411 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.411 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20167e10 con 0x7f5a200d0d90 2024-09-21T07:26:09.411 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20070dd0 con 0x7f5a200744f0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a279e4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5a20167f90 con 0x7f5a2006ffd0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a25f81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a25f81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50684/0 (socket says 172.21.15.105:50684) 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a25f81640 1 -- 172.21.15.105:0/534418460 learned_addr learned my addr 172.21.15.105:0/534418460 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a24f7f640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a200744f0 0x7f5a200b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.408+0000 7f5a25780640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2006ffd0 0x7f5a200b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 -- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2006ffd0 msgr2=0x7f5a200b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2006ffd0 0x7f5a200b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 -- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a200744f0 msgr2=0x7f5a200b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a200744f0 0x7f5a200b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 -- 172.21.15.105:0/534418460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a20154dd0 con 0x7f5a200d0d90 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a25f81640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c700 tx=0x7f5a1800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003b70 con 0x7f5a200d0d90 2024-09-21T07:26:09.412 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a18003d10 con 0x7f5a200d0d90 2024-09-21T07:26:09.413 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800aab0 con 0x7f5a200d0d90 2024-09-21T07:26:09.413 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a20155070 con 0x7f5a200d0d90 2024-09-21T07:26:09.413 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.409+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a20155630 con 0x7f5a200d0d90 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.410+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a200ce800 con 0x7f5a200d0d90 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.411+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a1800ac50 con 0x7f5a200d0d90 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.411+0000 7f5a15ffb640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 0x7f5a0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.411+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f5a1809a5d0 con 0x7f5a200d0d90 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.411+0000 7f5a25780640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 0x7f5a0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.414 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.412+0000 7f5a25780640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 0x7f5a0407b0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c001720 tx=0x7f5a1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.418 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.416+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a180627a0 con 0x7f5a200d0d90 2024-09-21T07:26:09.596 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.594+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a200cb220 con 0x7f5a200d0d90 2024-09-21T07:26:09.597 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.595+0000 7f5a15ffb640 1 -- 172.21.15.105:0/534418460 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12968 (secure 0 0 0) 0x7f5a180671d0 con 0x7f5a200d0d90 2024-09-21T07:26:09.597 INFO:teuthology.misc.health.smithi105.stdout: 2024-09-21T07:26:09.597 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":18,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:07.322181+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.597+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 msgr2=0x7f5a0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.597+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 0x7f5a0407b0d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c001720 tx=0x7f5a1c009040 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 msgr2=0x7f5a200b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c700 tx=0x7f5a1800cbc0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 shutdown_connections 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a04078bc0 0x7f5a0407b0d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a200744f0 0x7f5a200b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a200d0d90 0x7f5a200b2290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 --2- 172.21.15.105:0/534418460 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2006ffd0 0x7f5a200b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 >> 172.21.15.105:0/534418460 conn(0x7f5a200cdce0 msgr2=0x7f5a2015a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 shutdown_connections 2024-09-21T07:26:09.600 INFO:teuthology.misc.health.smithi105.stderr:2024-09-21T07:26:09.598+0000 7f5a279e4640 1 -- 172.21.15.105:0/534418460 wait complete. 2024-09-21T07:26:09.609 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T07:26:09.609 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T07:26:09.610 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:26:09.610 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:26:09.612 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:26:09.612 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:26:09.612 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:26:09.614 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:26:09.614 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:26:09.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:26:09.617 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:26:09.617 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:26:09.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:26:09.620 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:26:09.620 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T07:26:09.620 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:09.697 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-09-21T07:26:09.697 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-09-21T07:26:09.716 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-09-21T07:26:09.716 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-09-21T07:26:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 -- 172.21.15.105:0/2005630188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c0b0100 msgr2=0x7ff42c0cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2005630188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c0b0100 0x7ff42c0cfbd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff42c1596b0 tx=0x7ff42802a560 comp rx=0 tx=0).stop 2024-09-21T07:26:09.739 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 -- 172.21.15.105:0/2005630188 shutdown_connections 2024-09-21T07:26:09.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2005630188 >> v2:172.21.15.112:3300/0 conn(0x7ff42c15e090 0x7ff42c15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2005630188 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c0b0100 0x7ff42c0cfbd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 -- 172.21.15.105:0/2005630188 >> 172.21.15.105:0/2005630188 conn(0x7ff42c06cf80 msgr2=0x7ff42c06d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 -- 172.21.15.105:0/2005630188 shutdown_connections 2024-09-21T07:26:09.740 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.737+0000 7ff4340c5640 1 -- 172.21.15.105:0/2005630188 wait complete. 2024-09-21T07:26:09.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.739+0000 7ff4340c5640 1 Processor -- start 2024-09-21T07:26:09.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.739+0000 7ff4340c5640 1 -- start start 2024-09-21T07:26:09.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff42c0b0100 0x7ff42c144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c0d0110 0x7ff42c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c15c720 con 0x7ff42c15e090 2024-09-21T07:26:09.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c15c5a0 con 0x7ff42c0d0110 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff4340c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff42c0cd6c0 con 0x7ff42c0b0100 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff432662640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff432662640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50732/0 (socket says 172.21.15.105:50732) 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff432662640 1 -- 172.21.15.105:0/2305850337 learned_addr learned my addr 172.21.15.105:0/2305850337 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff431e61640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff42c0b0100 0x7ff42c144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.740+0000 7ff431660640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c0d0110 0x7ff42c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 -- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff42c0b0100 msgr2=0x7ff42c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff42c0b0100 0x7ff42c144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 -- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c0d0110 msgr2=0x7ff42c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c0d0110 0x7ff42c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 -- 172.21.15.105:0/2305850337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c158640 con 0x7ff42c15e090 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff432662640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e990 tx=0x7ff42400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424004d20 con 0x7ff42c15e090 2024-09-21T07:26:09.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff42400caa0 con 0x7ff42c15e090 2024-09-21T07:26:09.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c158920 con 0x7ff42c15e090 2024-09-21T07:26:09.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424005020 con 0x7ff42c15e090 2024-09-21T07:26:09.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.741+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c158ee0 con 0x7ff42c15e090 2024-09-21T07:26:09.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.742+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c1583b0 con 0x7ff42c15e090 2024-09-21T07:26:09.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.743+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4240051c0 con 0x7ff42c15e090 2024-09-21T07:26:09.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.743+0000 7ff4227fc640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 0x7ff41007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:09.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.743+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff42409b2b0 con 0x7ff42c15e090 2024-09-21T07:26:09.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.743+0000 7ff431e61640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 0x7ff41007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:09.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.744+0000 7ff431e61640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 0x7ff41007b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff428033000 tx=0x7ff428004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:09.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.748+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff424016030 con 0x7ff42c15e090 2024-09-21T07:26:09.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.961+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff42c0cfc00 con 0x7ff42c15e090 2024-09-21T07:26:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.962+0000 7ff4227fc640 1 -- 172.21.15.105:0/2305850337 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7ff42c0cfc00 con 0x7ff42c15e090 2024-09-21T07:26:09.964 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:09.964 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"btime":"2024-09-21T07:26:09:700916+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T07:26:09.964 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 2 2024-09-21T07:26:09.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 msgr2=0x7ff41007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 0x7ff41007b0d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff428033000 tx=0x7ff428004730 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 msgr2=0x7ff42c1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e990 tx=0x7ff42400ee50 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 shutdown_connections 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff410078bc0 0x7ff41007b0d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff42c0d0110 0x7ff42c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff42c15e090 0x7ff42c1596e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 --2- 172.21.15.105:0/2305850337 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff42c0b0100 0x7ff42c144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 >> 172.21.15.105:0/2305850337 conn(0x7ff42c06cf80 msgr2=0x7ff42c15cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 shutdown_connections 2024-09-21T07:26:09.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:09.964+0000 7ff4340c5640 1 -- 172.21.15.105:0/2305850337 wait complete. 2024-09-21T07:26:11.978 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 -- 172.21.15.105:0/2385858409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cd090 msgr2=0x7ff8a80cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 --2- 172.21.15.105:0/2385858409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cd090 0x7ff8a80cd4a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009830 tx=0x7ff89c02a510 comp rx=0 tx=0).stop 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 -- 172.21.15.105:0/2385858409 shutdown_connections 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 --2- 172.21.15.105:0/2385858409 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cd090 0x7ff8a80cd4a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 --2- 172.21.15.105:0/2385858409 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cff70 0x7ff8a80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.101+0000 7ff8ae950640 1 -- 172.21.15.105:0/2385858409 >> 172.21.15.105:0/2385858409 conn(0x7ff8a80cee20 msgr2=0x7ff8a80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.102+0000 7ff8ae950640 1 -- 172.21.15.105:0/2385858409 shutdown_connections 2024-09-21T07:26:12.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.102+0000 7ff8ae950640 1 -- 172.21.15.105:0/2385858409 wait complete. 2024-09-21T07:26:12.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.103+0000 7ff8ae950640 1 Processor -- start 2024-09-21T07:26:12.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.104+0000 7ff8ae950640 1 -- start start 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cd9e0 0x7ff8a81f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cff70 0x7ff8a809f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a80b7950 con 0x7ff8a80cd090 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a80b77d0 con 0x7ff8a80cff70 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8ae950640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a80cddf0 con 0x7ff8a80cd9e0 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47006/0 (socket says 172.21.15.105:47006) 2024-09-21T07:26:12.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 -- 172.21.15.105:0/3629575235 learned_addr learned my addr 172.21.15.105:0/3629575235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cd9e0 0x7ff8a81f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8aceed640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cff70 0x7ff8a809f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 -- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cd9e0 msgr2=0x7ff8a81f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cd9e0 0x7ff8a81f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 -- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cff70 msgr2=0x7ff8a809f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cff70 0x7ff8a809f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 -- 172.21.15.105:0/3629575235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a80cb1b0 con 0x7ff8a80cd090 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.105+0000 7ff8a7fff640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9f0 tx=0x7ff89800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.106+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89800cd80 con 0x7ff8a80cd090 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.106+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff898004020 con 0x7ff8a80cd090 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.106+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a80ca3f0 con 0x7ff8a80cd090 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.106+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898004470 con 0x7ff8a80cd090 2024-09-21T07:26:12.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.106+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a80ca9b0 con 0x7ff8a80cd090 2024-09-21T07:26:12.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.107+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a809bba0 con 0x7ff8a80cd090 2024-09-21T07:26:12.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.107+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8980035f0 con 0x7ff8a80cd090 2024-09-21T07:26:12.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.108+0000 7ff8a4ff9640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.108+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff89809b210 con 0x7ff8a80cd090 2024-09-21T07:26:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.108+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.109+0000 7ff8a77fe640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 0x7ff89407b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff89c034000 tx=0x7ff89c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.113+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff898063350 con 0x7ff8a80cd090 2024-09-21T07:26:12.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:26:12.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:26:12.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.324+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8a80b81d0 con 0x7ff8a80cd090 2024-09-21T07:26:12.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.325+0000 7ff8a4ff9640 1 -- 172.21.15.105:0/3629575235 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3918 (secure 0 0 0) 0x7ff898067d80 con 0x7ff8a80cd090 2024-09-21T07:26:12.327 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:12.327 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"btime":"2024-09-21T07:26:10:701932+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T07:26:12.327 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 msgr2=0x7ff89407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 0x7ff89407b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff89c034000 tx=0x7ff89c004730 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 msgr2=0x7ff8a81e6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff89800e9f0 tx=0x7ff89800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 shutdown_connections 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff894078b90 0x7ff89407b0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff8a80cff70 0x7ff8a809f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff8a80cd9e0 0x7ff8a81f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 --2- 172.21.15.105:0/3629575235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff8a80cd090 0x7ff8a81e6cf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 >> 172.21.15.105:0/3629575235 conn(0x7ff8a80cee20 msgr2=0x7ff8a81e4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 shutdown_connections 2024-09-21T07:26:12.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.328+0000 7ff8ae950640 1 -- 172.21.15.105:0/3629575235 wait complete. 2024-09-21T07:26:12.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 -- 172.21.15.105:0/3599158769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80d0a30 msgr2=0x7fb1b80d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 --2- 172.21.15.105:0/3599158769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80d0a30 0x7fb1b80d0e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005fd0 tx=0x7fb1b002a8a0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 -- 172.21.15.105:0/3599158769 shutdown_connections 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 --2- 172.21.15.105:0/3599158769 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80d0a30 0x7fb1b80d0e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 --2- 172.21.15.105:0/3599158769 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80cff00 0x7fb1b80d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 -- 172.21.15.105:0/3599158769 >> 172.21.15.105:0/3599158769 conn(0x7fb1b80cedb0 msgr2=0x7fb1b80cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 -- 172.21.15.105:0/3599158769 shutdown_connections 2024-09-21T07:26:12.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.513+0000 7fb1befe6640 1 -- 172.21.15.105:0/3599158769 wait complete. 2024-09-21T07:26:12.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.515+0000 7fb1befe6640 1 Processor -- start 2024-09-21T07:26:12.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.515+0000 7fb1befe6640 1 -- start start 2024-09-21T07:26:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80b7940 0x7fb1b81f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80d0a30 0x7fb1b814d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80cebe0 con 0x7fb1b80cff00 2024-09-21T07:26:12.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80cea60 con 0x7fb1b80b7940 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1befe6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80ccf40 con 0x7fb1b80d0a30 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47020/0 (socket says 172.21.15.105:47020) 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.516+0000 7fb1b7fff640 1 -- 172.21.15.105:0/2655324001 learned_addr learned my addr 172.21.15.105:0/2655324001 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1bcd82640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80b7940 0x7fb1b81f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1bd583640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80d0a30 0x7fb1b814d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 -- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80d0a30 msgr2=0x7fb1b814d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80d0a30 0x7fb1b814d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 -- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80b7940 msgr2=0x7fb1b81f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80b7940 0x7fb1b81f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 -- 172.21.15.105:0/2655324001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b815ed30 con 0x7fb1b80cff00 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b7fff640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e940 tx=0x7fb1ac00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00cd20 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1ac010430 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b815f010 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac010880 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.517+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b815f5d0 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.518+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b80d0310 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.519+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1ac010a20 con 0x7fb1b80cff00 2024-09-21T07:26:12.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.520+0000 7fb1b57fa640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 0x7fb1a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.520+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fb1ac09b470 con 0x7fb1b80cff00 2024-09-21T07:26:12.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.520+0000 7fb1bcd82640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 0x7fb1a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.520+0000 7fb1bcd82640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 0x7fb1a407b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007fd0 tx=0x7fb1a8001370 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.524+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1ac063530 con 0x7fb1b80cff00 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.707+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1b80cf9e0 con 0x7fb1b80cff00 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.707+0000 7fb1b57fa640 1 -- 172.21.15.105:0/2655324001 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fb1ac067f60 con 0x7fb1b80cff00 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-09-21T07:26:12.709 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:26:12.710 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.709+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 msgr2=0x7fb1a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.709+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 0x7fb1a407b0a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007fd0 tx=0x7fb1a8001370 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.709+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 msgr2=0x7fb1b81f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.709+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e940 tx=0x7fb1ac00ee00 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 shutdown_connections 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb1a4078b90 0x7fb1a407b0a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1b80d0a30 0x7fb1b814d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb1b80cff00 0x7fb1b81f3120 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 --2- 172.21.15.105:0/2655324001 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1b80b7940 0x7fb1b81f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 >> 172.21.15.105:0/2655324001 conn(0x7fb1b80cedb0 msgr2=0x7fb1b815c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 shutdown_connections 2024-09-21T07:26:12.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.710+0000 7fb1befe6640 1 -- 172.21.15.105:0/2655324001 wait complete. 2024-09-21T07:26:12.723 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:26:12.723 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:26:12.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 -- 172.21.15.105:0/828156728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac15e400 msgr2=0x7fccac1554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 --2- 172.21.15.105:0/828156728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac15e400 0x7fccac1554a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009830 tx=0x7fcca402a510 comp rx=0 tx=0).stop 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 -- 172.21.15.105:0/828156728 shutdown_connections 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 --2- 172.21.15.105:0/828156728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac15e400 0x7fccac1554a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 -- 172.21.15.105:0/828156728 >> 172.21.15.105:0/828156728 conn(0x7fccac0b6600 msgr2=0x7fccac0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 -- 172.21.15.105:0/828156728 shutdown_connections 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.838+0000 7fccb3016640 1 -- 172.21.15.105:0/828156728 wait complete. 2024-09-21T07:26:12.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.840+0000 7fccb3016640 1 Processor -- start 2024-09-21T07:26:12.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.840+0000 7fccb3016640 1 -- start start 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 0x7fccac09ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccac15f320 0x7fccac14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fccac156060 con 0x7fccac0b4af0 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fccac155ee0 con 0x7fccac15e400 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb3016640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fccac1561e0 con 0x7fccac15f320 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb0db2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb0db2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47030/0 (socket says 172.21.15.105:47030) 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.841+0000 7fccb0db2640 1 -- 172.21.15.105:0/1984828570 learned_addr learned my addr 172.21.15.105:0/1984828570 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccabfff640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 0x7fccac09ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb15b3640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccac15f320 0x7fccac14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 -- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccac15f320 msgr2=0x7fccac14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccac15f320 0x7fccac14c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 -- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 msgr2=0x7fccac09ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 0x7fccac09ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 -- 172.21.15.105:0/1984828570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac0cfa20 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccabfff640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 0x7fccac09ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb0db2640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b9f0 tx=0x7fcc9c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00cbd0 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc9c003970 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac0cff20 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00ce60 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.842+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac0d04e0 con 0x7fccac0b4af0 2024-09-21T07:26:12.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.843+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac1554a0 con 0x7fccac0b4af0 2024-09-21T07:26:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.844+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcc9c014440 con 0x7fccac0b4af0 2024-09-21T07:26:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.844+0000 7fcca97fa640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 0x7fcc9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.844+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fcc9c09a910 con 0x7fccac0b4af0 2024-09-21T07:26:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.845+0000 7fccabfff640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 0x7fcc9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:12.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.845+0000 7fccabfff640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 0x7fcc9807b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcca0000f60 tx=0x7fcca0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:12.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:12.849+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcc9c063020 con 0x7fccac0b4af0 2024-09-21T07:26:13.027 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.025+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fccac0af1c0 con 0x7fccac0b4af0 2024-09-21T07:26:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.720+0000 7fcca97fa640 1 -- 172.21.15.105:0/1984828570 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fcc9c062920 con 0x7fccac0b4af0 2024-09-21T07:26:13.722 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-09-21T07:26:13.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.722+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 msgr2=0x7fcc9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.722+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 0x7fcc9807b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcca0000f60 tx=0x7fcca0009040 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.722+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 msgr2=0x7fccac0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.722+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b9f0 tx=0x7fcc9c00beb0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 shutdown_connections 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcc98078b90 0x7fcc9807b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fccac15e400 0x7fccac09ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fccac15f320 0x7fccac14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 --2- 172.21.15.105:0/1984828570 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fccac0b4af0 0x7fccac0aff70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 >> 172.21.15.105:0/1984828570 conn(0x7fccac0b6600 msgr2=0x7fccac0714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 shutdown_connections 2024-09-21T07:26:13.725 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.723+0000 7fccb3016640 1 -- 172.21.15.105:0/1984828570 wait complete. 2024-09-21T07:26:13.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-21T07:26:13.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 -- 172.21.15.105:0/1581695563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 msgr2=0x7f2d48003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 --2- 172.21.15.105:0/1581695563 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d48003ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001710 tx=0x7f2d48005830 comp rx=0 tx=0).stop 2024-09-21T07:26:13.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d61e00640 1 -- 172.21.15.105:0/1581695563 reap_dead start 2024-09-21T07:26:13.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 -- 172.21.15.105:0/1581695563 shutdown_connections 2024-09-21T07:26:13.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 -- 172.21.15.105:0/1581695563 >> 172.21.15.105:0/1581695563 conn(0x7f2d5c08bfb0 msgr2=0x7f2d5c09d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:13.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 -- 172.21.15.105:0/1581695563 shutdown_connections 2024-09-21T07:26:13.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.861+0000 7f2d63863640 1 -- 172.21.15.105:0/1581695563 wait complete. 2024-09-21T07:26:13.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.863+0000 7f2d63863640 1 Processor -- start 2024-09-21T07:26:13.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.863+0000 7f2d63863640 1 -- start start 2024-09-21T07:26:13.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 0x7f2d5c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 0x7f2d5c1e9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c0b5fe0 con 0x7f2d48003a30 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c0d1c30 con 0x7f2d5c15a740 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d63863640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5c0b6160 con 0x7f2d5c1e9700 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d615ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d615ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47048/0 (socket says 172.21.15.105:47048) 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.864+0000 7f2d615ff640 1 -- 172.21.15.105:0/4164005713 learned_addr learned my addr 172.21.15.105:0/4164005713 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d60dfe640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 0x7f2d5c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d61e00640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 0x7f2d5c1e9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 -- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 msgr2=0x7f2d5c1e9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 0x7f2d5c1e9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 -- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 msgr2=0x7f2d5c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 0x7f2d5c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:13.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 -- 172.21.15.105:0/4164005713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c1ea050 con 0x7f2d48003a30 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d60dfe640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 0x7f2d5c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d61e00640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 0x7f2d5c1e9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d615ff640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001800 tx=0x7f2d58006310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58041070 con 0x7f2d48003a30 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d58003850 con 0x7f2d48003a30 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58038070 con 0x7f2d48003a30 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c14bf60 con 0x7f2d48003a30 2024-09-21T07:26:13.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.865+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c15c880 con 0x7f2d48003a30 2024-09-21T07:26:13.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.866+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c0b66f0 con 0x7f2d48003a30 2024-09-21T07:26:13.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.867+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d58044070 con 0x7f2d48003a30 2024-09-21T07:26:13.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.867+0000 7f2d51ffb640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 0x7f2d4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:13.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.868+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f2d580ba260 con 0x7f2d48003a30 2024-09-21T07:26:13.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.868+0000 7f2d60dfe640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 0x7f2d4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:13.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.868+0000 7f2d60dfe640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 0x7f2d4007b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c162550 tx=0x7f2d4c0067a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:13.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:13.872+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d5c1ea050 con 0x7f2d48003a30 2024-09-21T07:26:14.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.052+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f2d5c0b1920 con 0x7f2d48003a30 2024-09-21T07:26:14.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.722+0000 7f2d51ffb640 1 -- 172.21.15.105:0/4164005713 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7f2d58082910 con 0x7f2d48003a30 2024-09-21T07:26:14.724 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-09-21T07:26:14.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.725+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 msgr2=0x7f2d4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.725+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 0x7f2d4007b0a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c162550 tx=0x7f2d4c0067a0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.725+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 msgr2=0x7f2d5c14ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.725+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d58001800 tx=0x7f2d58006310 comp rx=0 tx=0).stop 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 shutdown_connections 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d40078b90 0x7f2d4007b0a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d5c15a740 0x7f2d5c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d5c1e9700 0x7f2d5c1e9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 --2- 172.21.15.105:0/4164005713 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d48003a30 0x7f2d5c14ba20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 >> 172.21.15.105:0/4164005713 conn(0x7f2d5c08bfb0 msgr2=0x7f2d5c0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:14.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.726+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 shutdown_connections 2024-09-21T07:26:14.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.727+0000 7f2d63863640 1 -- 172.21.15.105:0/4164005713 wait complete. 2024-09-21T07:26:14.754 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:26:14.870 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 -- 172.21.15.105:0/4256155337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 msgr2=0x7fed8c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 --2- 172.21.15.105:0/4256155337 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7fed8c004c10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed98009b50 tx=0x7fed8c005830 comp rx=0 tx=0).stop 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda359e640 1 -- 172.21.15.105:0/4256155337 reap_dead start 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 -- 172.21.15.105:0/4256155337 shutdown_connections 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 -- 172.21.15.105:0/4256155337 >> 172.21.15.105:0/4256155337 conn(0x7feda409d850 msgr2=0x7feda408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 -- 172.21.15.105:0/4256155337 shutdown_connections 2024-09-21T07:26:14.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.868+0000 7feda91da640 1 -- 172.21.15.105:0/4256155337 wait complete. 2024-09-21T07:26:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.870+0000 7feda91da640 1 Processor -- start 2024-09-21T07:26:14.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.871+0000 7feda91da640 1 -- start start 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feda40cc240 0x7feda40d4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feda40cc650 0x7feda40d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7feda406f560 con 0x7fed8c004800 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feda406f3e0 con 0x7feda40cc650 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda91da640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7feda40d0cf0 con 0x7feda40cc240 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47066/0 (socket says 172.21.15.105:47066) 2024-09-21T07:26:14.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 -- 172.21.15.105:0/937930599 learned_addr learned my addr 172.21.15.105:0/937930599 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda359e640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feda40cc650 0x7feda40d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda259c640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feda40cc240 0x7feda40d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 -- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feda40cc240 msgr2=0x7feda40d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feda40cc240 0x7feda40d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 -- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feda40cc650 msgr2=0x7feda40d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feda40cc650 0x7feda40d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 -- 172.21.15.105:0/937930599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41f54c0 con 0x7fed8c004800 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.872+0000 7feda2d9d640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed94003df0 tx=0x7fed940017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.873+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94011640 con 0x7fed8c004800 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.873+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed94016020 con 0x7fed8c004800 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.873+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda41f57a0 con 0x7fed8c004800 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.873+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94011aa0 con 0x7fed8c004800 2024-09-21T07:26:14.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.873+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda41f5d60 con 0x7fed8c004800 2024-09-21T07:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.874+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda40d1220 con 0x7fed8c004800 2024-09-21T07:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.875+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed94011c40 con 0x7fed8c004800 2024-09-21T07:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.875+0000 7fed8b7fe640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 0x7fed9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.875+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fed9409b9b0 con 0x7fed8c004800 2024-09-21T07:26:14.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.875+0000 7feda259c640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 0x7fed9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:14.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.876+0000 7feda259c640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 0x7fed9007b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed98035000 tx=0x7fed9802d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:14.882 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:14.880+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed94020020 con 0x7fed8c004800 2024-09-21T07:26:15.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.093+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feda40b86a0 con 0x7fed8c004800 2024-09-21T07:26:15.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.753+0000 7fed8b7fe640 1 -- 172.21.15.105:0/937930599 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fed94063dd0 con 0x7fed8c004800 2024-09-21T07:26:15.756 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T07:26:15.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 msgr2=0x7fed9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 0x7fed9007b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed98035000 tx=0x7fed9802d040 comp rx=0 tx=0).stop 2024-09-21T07:26:15.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 msgr2=0x7feda41f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed94003df0 tx=0x7fed940017f0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 shutdown_connections 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fed90078bc0 0x7fed9007b0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feda40cc650 0x7feda40d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feda40cc240 0x7feda40d4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.760+0000 7feda91da640 1 --2- 172.21.15.105:0/937930599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fed8c004800 0x7feda41f8fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.761+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 >> 172.21.15.105:0/937930599 conn(0x7feda409d850 msgr2=0x7feda4161980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.761+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 shutdown_connections 2024-09-21T07:26:15.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.763+0000 7feda91da640 1 -- 172.21.15.105:0/937930599 wait complete. 2024-09-21T07:26:15.784 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-21T07:26:15.785 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-21T07:26:15.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 -- 172.21.15.105:0/446672312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 msgr2=0x7fb52c0d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 --2- 172.21.15.105:0/446672312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0d0e90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009830 tx=0x7fb51c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 -- 172.21.15.105:0/446672312 shutdown_connections 2024-09-21T07:26:15.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 --2- 172.21.15.105:0/446672312 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0d0e90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 -- 172.21.15.105:0/446672312 >> 172.21.15.105:0/446672312 conn(0x7fb52c0cedb0 msgr2=0x7fb52c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:15.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 -- 172.21.15.105:0/446672312 shutdown_connections 2024-09-21T07:26:15.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.903+0000 7fb5332a4640 1 -- 172.21.15.105:0/446672312 wait complete. 2024-09-21T07:26:15.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.905+0000 7fb5332a4640 1 Processor -- start 2024-09-21T07:26:15.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.905+0000 7fb5332a4640 1 -- start start 2024-09-21T07:26:15.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb52c0b7960 0x7fb52c0b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c0cff00 0x7fb52c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c0d0310 con 0x7fb52c0d0a80 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c0b7eb0 con 0x7fb52c0cff00 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb5332a4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb52c0d0490 con 0x7fb52c0b7960 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb531841640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb531841640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47078/0 (socket says 172.21.15.105:47078) 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.906+0000 7fb531841640 1 -- 172.21.15.105:0/3057828829 learned_addr learned my addr 172.21.15.105:0/3057828829 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb53083f640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c0cff00 0x7fb52c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531040640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb52c0b7960 0x7fb52c0b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 -- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb52c0b7960 msgr2=0x7fb52c0b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb52c0b7960 0x7fb52c0b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 -- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c0cff00 msgr2=0x7fb52c1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c0cff00 0x7fb52c1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 -- 172.21.15.105:0/3057828829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c0cb130 con 0x7fb52c0d0a80 2024-09-21T07:26:15.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb531841640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c6b0 tx=0x7fb52400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524003b70 con 0x7fb52c0d0a80 2024-09-21T07:26:15.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb524003d30 con 0x7fb52c0d0a80 2024-09-21T07:26:15.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c0cb670 con 0x7fb52c0d0a80 2024-09-21T07:26:15.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524005330 con 0x7fb52c0d0a80 2024-09-21T07:26:15.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.907+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c0cbc30 con 0x7fb52c0d0a80 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.908+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52c0d0e90 con 0x7fb52c0d0a80 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.909+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5240054d0 con 0x7fb52c0d0a80 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.910+0000 7fb521ffb640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 0x7fb51007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.910+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb52409aa00 con 0x7fb52c0d0a80 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.910+0000 7fb531040640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 0x7fb51007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:15.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.910+0000 7fb531040640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 0x7fb51007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb528007fd0 tx=0x7fb528001370 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:15.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:15.914+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5240627f0 con 0x7fb52c0d0a80 2024-09-21T07:26:16.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.101+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fb52c0af470 con 0x7fb52c0d0a80 2024-09-21T07:26:16.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.755+0000 7fb521ffb640 1 -- 172.21.15.105:0/3057828829 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7fb524067220 con 0x7fb52c0d0a80 2024-09-21T07:26:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 msgr2=0x7fb51007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 0x7fb51007b0a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb528007fd0 tx=0x7fb528001370 comp rx=0 tx=0).stop 2024-09-21T07:26:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 msgr2=0x7fb52c0b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb52400c6b0 tx=0x7fb52400cb70 comp rx=0 tx=0).stop 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 shutdown_connections 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb510078b90 0x7fb51007b0a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb52c0cff00 0x7fb52c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb52c0d0a80 0x7fb52c0b5160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 --2- 172.21.15.105:0/3057828829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb52c0b7960 0x7fb52c0b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 >> 172.21.15.105:0/3057828829 conn(0x7fb52c0cedb0 msgr2=0x7fb52c15cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 shutdown_connections 2024-09-21T07:26:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.758+0000 7fb5332a4640 1 -- 172.21.15.105:0/3057828829 wait complete. 2024-09-21T07:26:16.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:26:16.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 -- 172.21.15.105:0/2885181101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 msgr2=0x7f57dc0b5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 --2- 172.21.15.105:0/2885181101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc0b5a40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009830 tx=0x7f57c402a480 comp rx=0 tx=0).stop 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 -- 172.21.15.105:0/2885181101 shutdown_connections 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 --2- 172.21.15.105:0/2885181101 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b5f80 0x7f57dc1653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 --2- 172.21.15.105:0/2885181101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc0b5a40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 -- 172.21.15.105:0/2885181101 >> 172.21.15.105:0/2885181101 conn(0x7f57dc0b2e50 msgr2=0x7f57dc0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 -- 172.21.15.105:0/2885181101 shutdown_connections 2024-09-21T07:26:16.895 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.892+0000 7f57e0820640 1 -- 172.21.15.105:0/2885181101 wait complete. 2024-09-21T07:26:16.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.894+0000 7f57e0820640 1 Processor -- start 2024-09-21T07:26:16.896 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.894+0000 7f57e0820640 1 -- start start 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 0x7f57dc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 0x7f57dc1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc165cb0 con 0x7f57dc0b5630 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc165b30 con 0x7f57dc0b5f80 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.895+0000 7f57e0820640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc165e30 con 0x7f57dc0b6590 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47092/0 (socket says 172.21.15.105:47092) 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 -- 172.21.15.105:0/2411384585 learned_addr learned my addr 172.21.15.105:0/2411384585 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57dad9d640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 0x7f57dc1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:16.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57d3fff640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 0x7f57dc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 -- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 msgr2=0x7f57dc1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 0x7f57dc1579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 -- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 msgr2=0x7f57dc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 0x7f57dc153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 -- 172.21.15.105:0/2411384585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc167fc0 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57dad9d640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 0x7f57dc1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57d3fff640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 0x7f57dc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57da59c640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c3c0 tx=0x7f57cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc017070 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57cc0035f0 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc0c6a80 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc003a20 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.896+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc0c7040 con 0x7f57dc0b5630 2024-09-21T07:26:16.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.897+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc0b5a40 con 0x7f57dc0b5630 2024-09-21T07:26:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.898+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57cc01a070 con 0x7f57dc0b5630 2024-09-21T07:26:16.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.899+0000 7f57d2ffd640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 0x7f57bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.899+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f57cc09a0d0 con 0x7f57dc0b5630 2024-09-21T07:26:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.899+0000 7f57d3fff640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 0x7f57bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:16.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.899+0000 7f57d3fff640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 0x7f57bc07b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57c4034000 tx=0x7f57c4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:16.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:16.903+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57cc0623e0 con 0x7f57dc0b5630 2024-09-21T07:26:17.081 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:17.080+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f57dc06f0c0 con 0x7f57dc0b5630 2024-09-21T07:26:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.759+0000 7f57d2ffd640 1 -- 172.21.15.105:0/2411384585 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f57cc061ce0 con 0x7f57dc0b5630 2024-09-21T07:26:18.761 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' created 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 msgr2=0x7f57bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 0x7f57bc07b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f57c4034000 tx=0x7f57c4004730 comp rx=0 tx=0).stop 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 msgr2=0x7f57dc167a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f57cc00c3c0 tx=0x7f57cc00c880 comp rx=0 tx=0).stop 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 shutdown_connections 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f57bc078b90 0x7f57bc07b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57dc0b5f80 0x7f57dc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57dc0b6590 0x7f57dc1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 --2- 172.21.15.105:0/2411384585 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57dc0b5630 0x7f57dc167a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.762+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 >> 172.21.15.105:0/2411384585 conn(0x7f57dc0b2e50 msgr2=0x7f57dc15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.763+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 shutdown_connections 2024-09-21T07:26:18.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.763+0000 7f57e0820640 1 -- 172.21.15.105:0/2411384585 wait complete. 2024-09-21T07:26:18.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-21T07:26:18.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 -- 172.21.15.105:0/1892713893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b3580 msgr2=0x7f0bfc0b9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:18.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 --2- 172.21.15.105:0/1892713893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b3580 0x7f0bfc0b9b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0cfe60 tx=0x7f0bf802a480 comp rx=0 tx=0).stop 2024-09-21T07:26:18.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 -- 172.21.15.105:0/1892713893 shutdown_connections 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 --2- 172.21.15.105:0/1892713893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b4ff0 0x7f0bfc0d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 --2- 172.21.15.105:0/1892713893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b3580 0x7f0bfc0b9b20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 -- 172.21.15.105:0/1892713893 >> 172.21.15.105:0/1892713893 conn(0x7f0bfc08bfb0 msgr2=0x7f0bfc0d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 -- 172.21.15.105:0/1892713893 shutdown_connections 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.903+0000 7f0c02efa640 1 -- 172.21.15.105:0/1892713893 wait complete. 2024-09-21T07:26:18.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.905+0000 7f0c02efa640 1 Processor -- start 2024-09-21T07:26:18.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.905+0000 7f0c02efa640 1 -- start start 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b3580 0x7f0bfc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b4ff0 0x7f0bfc1e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0d2d30 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0d2bb0 con 0x7f0bfc0b4ff0 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c02efa640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc0d2eb0 con 0x7f0bfc0b3580 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c01497640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:18.910 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c01497640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47102/0 (socket says 172.21.15.105:47102) 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.906+0000 7f0c01497640 1 -- 172.21.15.105:0/3455732435 learned_addr learned my addr 172.21.15.105:0/3455732435 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0bf3fff640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b4ff0 0x7f0bfc1e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c00c96640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b3580 0x7f0bfc150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 -- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b3580 msgr2=0x7f0bfc150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b3580 0x7f0bfc150e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 -- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b4ff0 msgr2=0x7f0bfc1e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b4ff0 0x7f0bfc1e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 -- 172.21.15.105:0/3455732435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0cc7c0 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c01497640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400ec20 tx=0x7f0bf400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4003b70 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc0ccaa0 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf4003d10 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004280 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.907+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc0cd060 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.908+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc0d3360 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.912 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.909+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0bf4004420 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.910+0000 7f0bf17fa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:18.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.910+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f0bf409bbe0 con 0x7f0bfc0d0fa0 2024-09-21T07:26:18.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.910+0000 7f0c00c96640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:18.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.910+0000 7f0c00c96640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 0x7f0be807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8034000 tx=0x7f0bf8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:18.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:18.915+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bf40636c0 con 0x7f0bfc0d0fa0 2024-09-21T07:26:19.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.093+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f0bfc0b5800 con 0x7f0bfc0d0fa0 2024-09-21T07:26:19.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.763+0000 7f0bf17fa640 1 -- 172.21.15.105:0/3455732435 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f0bf40680f0 con 0x7f0bfc0d0fa0 2024-09-21T07:26:19.766 INFO:teuthology.orchestra.run.smithi105.stderr:set pool 4 allow_ec_overwrites to true 2024-09-21T07:26:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.767+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 msgr2=0x7f0be807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:19.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.767+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 0x7f0be807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8034000 tx=0x7f0bf8004730 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.767+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 msgr2=0x7f0bfc14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.767+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400ec20 tx=0x7f0bf400c6a0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 shutdown_connections 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0be8078b90 0x7f0be807b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b4ff0 0x7f0bfc1e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0bfc0d0fa0 0x7f0bfc14e210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 --2- 172.21.15.105:0/3455732435 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0bfc0b3580 0x7f0bfc150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.768+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 >> 172.21.15.105:0/3455732435 conn(0x7f0bfc08bfb0 msgr2=0x7f0bfc161a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.769+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 shutdown_connections 2024-09-21T07:26:19.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.769+0000 7f0c02efa640 1 -- 172.21.15.105:0/3455732435 wait complete. 2024-09-21T07:26:19.801 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 -- 172.21.15.105:0/3494919296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e00701e0 msgr2=0x7fa5e015c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 --2- 172.21.15.105:0/3494919296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e00701e0 0x7fa5e015c670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0072b20 tx=0x7fa5dc02a530 comp rx=0 tx=0).stop 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 -- 172.21.15.105:0/3494919296 shutdown_connections 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 --2- 172.21.15.105:0/3494919296 >> v2:172.21.15.112:3300/0 conn(0x7fa5e0074120 0x7fa5e0074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 --2- 172.21.15.105:0/3494919296 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e00701e0 0x7fa5e015c670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.913+0000 7fa5e735e640 1 -- 172.21.15.105:0/3494919296 >> 172.21.15.105:0/3494919296 conn(0x7fa5e00cdce0 msgr2=0x7fa5e00ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.914+0000 7fa5e735e640 1 -- 172.21.15.105:0/3494919296 shutdown_connections 2024-09-21T07:26:19.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.914+0000 7fa5e735e640 1 -- 172.21.15.105:0/3494919296 wait complete. 2024-09-21T07:26:19.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.915+0000 7fa5e735e640 1 Processor -- start 2024-09-21T07:26:19.917 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.915+0000 7fa5e735e640 1 -- start start 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5e00701e0 0x7fa5e00b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5e0074120 0x7fa5e00b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0074bf0 con 0x7fa5e006cda0 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e0074a70 con 0x7fa5e00701e0 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e0074d70 con 0x7fa5e0074120 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:47120/0 (socket says 172.21.15.105:47120) 2024-09-21T07:26:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 -- 172.21.15.105:0/2980836494 learned_addr learned my addr 172.21.15.105:0/2980836494 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e48f9640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5e00701e0 0x7fa5e00b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e58fb640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5e0074120 0x7fa5e00b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 -- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5e0074120 msgr2=0x7fa5e00b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5e0074120 0x7fa5e00b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 -- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5e00701e0 msgr2=0x7fa5e00b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5e00701e0 0x7fa5e00b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 -- 172.21.15.105:0/2980836494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e00c8490 con 0x7fa5e006cda0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e50fa640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009920 tx=0x7fa5dc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc005c90 con 0x7fa5e006cda0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5dc031e50 con 0x7fa5e006cda0 2024-09-21T07:26:19.921 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5e00c8770 con 0x7fa5e006cda0 2024-09-21T07:26:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.917+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc005240 con 0x7fa5e006cda0 2024-09-21T07:26:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.918+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5e00c8d30 con 0x7fa5e006cda0 2024-09-21T07:26:19.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.918+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5e00d23b0 con 0x7fa5e006cda0 2024-09-21T07:26:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.919+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa5dc0053e0 con 0x7fa5e006cda0 2024-09-21T07:26:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.920+0000 7fa5d5ffb640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 0x7fa5c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.920+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fa5dc0bc190 con 0x7fa5e006cda0 2024-09-21T07:26:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.920+0000 7fa5e48f9640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 0x7fa5c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.920+0000 7fa5e48f9640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 0x7fa5c407b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0000f60 tx=0x7fa5d0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:19.924+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5dc03f030 con 0x7fa5e006cda0 2024-09-21T07:26:20.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.138+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fa5e0075220 con 0x7fa5e006cda0 2024-09-21T07:26:20.714 INFO:tasks.ceph.mds.b.smithi112.stderr:2024-09-21T07:26:20.712+0000 7fb2d985c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:26:20.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.795+0000 7fa5d5ffb640 1 -- 172.21.15.105:0/2980836494 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fa5dc031770 con 0x7fa5e006cda0 2024-09-21T07:26:20.797 INFO:teuthology.orchestra.run.smithi105.stderr:added data pool 4 to fsmap 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 msgr2=0x7fa5c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 0x7fa5c407b0d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0000f60 tx=0x7fa5d0009040 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 msgr2=0x7fa5e015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009920 tx=0x7fa5dc033c20 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 shutdown_connections 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa5c4078bc0 0x7fa5c407b0d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5e00701e0 0x7fa5e00b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5e0074120 0x7fa5e00b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 --2- 172.21.15.105:0/2980836494 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5e006cda0 0x7fa5e015aaa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 >> 172.21.15.105:0/2980836494 conn(0x7fa5e00cdce0 msgr2=0x7fa5e0068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 shutdown_connections 2024-09-21T07:26:20.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.798+0000 7fa5e735e640 1 -- 172.21.15.105:0/2980836494 wait complete. 2024-09-21T07:26:20.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 -- 172.21.15.105:0/2427979455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80480b6440 msgr2=0x7f80480cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 --2- 172.21.15.105:0/2427979455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80480b6440 0x7f80480cf050 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f804808e330 tx=0x7f803802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 -- 172.21.15.105:0/2427979455 shutdown_connections 2024-09-21T07:26:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 --2- 172.21.15.105:0/2427979455 >> v2:172.21.15.112:3300/0 conn(0x7f80480b6850 0x7f80480cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 --2- 172.21.15.105:0/2427979455 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f80480b6440 0x7f80480cf050 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 -- 172.21.15.105:0/2427979455 >> 172.21.15.105:0/2427979455 conn(0x7f804808bfb0 msgr2=0x7f80480d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:20.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 -- 172.21.15.105:0/2427979455 shutdown_connections 2024-09-21T07:26:20.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.929+0000 7f804fa84640 1 -- 172.21.15.105:0/2427979455 wait complete. 2024-09-21T07:26:20.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.931+0000 7f804fa84640 1 Processor -- start 2024-09-21T07:26:20.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.931+0000 7f804fa84640 1 -- start start 2024-09-21T07:26:20.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80480b6440 0x7f804815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80480b6850 0x7f80480cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f80480cfc50 con 0x7f804808c3c0 2024-09-21T07:26:20.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80480cfad0 con 0x7f80480b6440 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804fa84640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8048162c70 con 0x7f80480b6850 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38510/0 (socket says 172.21.15.105:38510) 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 -- 172.21.15.105:0/2242222785 learned_addr learned my addr 172.21.15.105:0/2242222785 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d01f640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80480b6440 0x7f804815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804e021640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80480b6850 0x7f80480cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 -- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80480b6850 msgr2=0x7f80480cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80480b6850 0x7f80480cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 -- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80480b6440 msgr2=0x7f804815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80480b6440 0x7f804815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.932+0000 7f804d820640 1 -- 172.21.15.105:0/2242222785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80480c8f80 con 0x7f804808c3c0 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f804d820640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f804400b3f0 tx=0x7f804400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044015020 con 0x7f804808c3c0 2024-09-21T07:26:20.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8044011920 con 0x7f804808c3c0 2024-09-21T07:26:20.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044011d70 con 0x7f804808c3c0 2024-09-21T07:26:20.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80481f92b0 con 0x7f804808c3c0 2024-09-21T07:26:20.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.933+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80480a1da0 con 0x7f804808c3c0 2024-09-21T07:26:20.937 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.934+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80480a2030 con 0x7f804808c3c0 2024-09-21T07:26:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.934+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8044018020 con 0x7f804808c3c0 2024-09-21T07:26:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.935+0000 7f803e7fc640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 0x7f802c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.935+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f804409aac0 con 0x7f804808c3c0 2024-09-21T07:26:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.935+0000 7f804d01f640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 0x7f802c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:20.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.936+0000 7f804d01f640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 0x7f802c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8038033000 tx=0x7f8038004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:20.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:20.940+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80480a2030 con 0x7f804808c3c0 2024-09-21T07:26:21.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.152+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804806f7f0 con 0x7f804808c3c0 2024-09-21T07:26:21.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.152+0000 7f803e7fc640 1 -- 172.21.15.105:0/2242222785 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5231 (secure 0 0 0) 0x7f804401f030 con 0x7f804808c3c0 2024-09-21T07:26:21.154 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:21.154 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"btime":"2024-09-21T07:26:20:784760+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:20.139726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:21.155 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2024-09-21T07:26:21.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 msgr2=0x7f802c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 0x7f802c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8038033000 tx=0x7f8038004730 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 msgr2=0x7f80480cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f804400b3f0 tx=0x7f804400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 shutdown_connections 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f802c078b90 0x7f802c07b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80480b6440 0x7f804815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f80480b6850 0x7f80480cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 --2- 172.21.15.105:0/2242222785 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f804808c3c0 0x7f80480cb1e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.154+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 >> 172.21.15.105:0/2242222785 conn(0x7f804808bfb0 msgr2=0x7f804815ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.155+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 shutdown_connections 2024-09-21T07:26:21.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.155+0000 7f804fa84640 1 -- 172.21.15.105:0/2242222785 wait complete. 2024-09-21T07:26:21.167 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:21.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 -- 172.21.15.105:0/2546992536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 msgr2=0x7faf4c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 --2- 172.21.15.105:0/2546992536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c06d560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7faf3c005fd0 tx=0x7faf3c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 -- 172.21.15.105:0/2546992536 shutdown_connections 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 --2- 172.21.15.105:0/2546992536 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c06d560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 -- 172.21.15.105:0/2546992536 >> 172.21.15.105:0/2546992536 conn(0x7faf4c0b6610 msgr2=0x7faf4c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 -- 172.21.15.105:0/2546992536 shutdown_connections 2024-09-21T07:26:21.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.278+0000 7faf50fe6640 1 -- 172.21.15.105:0/2546992536 wait complete. 2024-09-21T07:26:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.280+0000 7faf50fe6640 1 Processor -- start 2024-09-21T07:26:21.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.280+0000 7faf50fe6640 1 -- start start 2024-09-21T07:26:21.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c0ae090 0x7faf4c141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf4c1f3c60 0x7faf4c1f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c15e310 con 0x7faf4c15a3a0 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf4c15e190 con 0x7faf4c0ae090 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.281+0000 7faf50fe6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7faf4c15e490 con 0x7faf4c1f3c60 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38520/0 (socket says 172.21.15.105:38520) 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 -- 172.21.15.105:0/984717748 learned_addr learned my addr 172.21.15.105:0/984717748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf4a59c640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c0ae090 0x7faf4c141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf4ad9d640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf4c1f3c60 0x7faf4c1f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 -- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf4c1f3c60 msgr2=0x7faf4c1f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf4c1f3c60 0x7faf4c1f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 -- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c0ae090 msgr2=0x7faf4c141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c0ae090 0x7faf4c141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 -- 172.21.15.105:0/984717748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf4c15dfc0 con 0x7faf4c15a3a0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf49d9b640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c3a0 tx=0x7faf4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40006890 con 0x7faf4c15a3a0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf40006b20 con 0x7faf4c15a3a0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4c1f4450 con 0x7faf4c15a3a0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40007020 con 0x7faf4c15a3a0 2024-09-21T07:26:21.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.282+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4c152fd0 con 0x7faf4c15a3a0 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.283+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf4c1f41c0 con 0x7faf4c15a3a0 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.284+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf40019020 con 0x7faf4c15a3a0 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.284+0000 7faf3affd640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 0x7faf2c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.285+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7faf4009a6b0 con 0x7faf4c15a3a0 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.285+0000 7faf4a59c640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 0x7faf2c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.285+0000 7faf4a59c640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 0x7faf2c07b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.289+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf400620e0 con 0x7faf4c15a3a0 2024-09-21T07:26:21.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.467+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf4c0af890 con 0x7faf4c15a3a0 2024-09-21T07:26:21.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.468+0000 7faf3affd640 1 -- 172.21.15.105:0/984717748 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18683 (secure 0 0 0) 0x7faf4c0af890 con 0x7faf4c15a3a0 2024-09-21T07:26:21.470 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:21.470 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:20.139487+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:21.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 msgr2=0x7faf2c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 0x7faf2c07b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf3c034000 tx=0x7faf3c004ab0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 msgr2=0x7faf4c1f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7faf4000c3a0 tx=0x7faf4000c860 comp rx=0 tx=0).stop 2024-09-21T07:26:21.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 shutdown_connections 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf2c078bc0 0x7faf2c07b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf4c1f3c60 0x7faf4c1f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf4c15a3a0 0x7faf4c1f3720 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 --2- 172.21.15.105:0/984717748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf4c0ae090 0x7faf4c141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 >> 172.21.15.105:0/984717748 conn(0x7faf4c0b6610 msgr2=0x7faf4c151010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 shutdown_connections 2024-09-21T07:26:21.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.470+0000 7faf50fe6640 1 -- 172.21.15.105:0/984717748 wait complete. 2024-09-21T07:26:21.483 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:21.595 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 -- 172.21.15.105:0/482599426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 msgr2=0x7f59280d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 --2- 172.21.15.105:0/482599426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59280d13d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f592808d450 tx=0x7f592002a980 comp rx=0 tx=0).stop 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 -- 172.21.15.105:0/482599426 shutdown_connections 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 --2- 172.21.15.105:0/482599426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59280d13d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 --2- 172.21.15.105:0/482599426 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 0x7f59280d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 --2- 172.21.15.105:0/482599426 >> v2:172.21.15.112:3300/0 conn(0x7f59280d3020 0x7f59280b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 -- 172.21.15.105:0/482599426 >> 172.21.15.105:0/482599426 conn(0x7f592808bfb0 msgr2=0x7f592809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 -- 172.21.15.105:0/482599426 shutdown_connections 2024-09-21T07:26:21.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.593+0000 7f593009b640 1 -- 172.21.15.105:0/482599426 wait complete. 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.594+0000 7f593009b640 1 Processor -- start 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.595+0000 7f593009b640 1 -- start start 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59280d3020 0x7f592815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 0x7f59281f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59280b4aa0 con 0x7f59280d0fc0 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59280b4920 con 0x7f59280d3020 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f593009b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f59280d1910 con 0x7f59280d4460 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38538/0 (socket says 172.21.15.105:38538) 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 -- 172.21.15.105:0/2711536235 learned_addr learned my addr 172.21.15.105:0/2711536235 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:21.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592d636640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59280d3020 0x7f592815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592e638640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 0x7f59281f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 -- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 msgr2=0x7f59281f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 0x7f59281f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 -- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59280d3020 msgr2=0x7f592815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59280d3020 0x7f592815ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 -- 172.21.15.105:0/2711536235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59281e80c0 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f592de37640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f592400d690 tx=0x7f592400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.596+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924004220 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.597+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5924009820 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.597+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924009c50 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.597+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59281f8f50 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.597+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59281f7270 con 0x7f59280d0fc0 2024-09-21T07:26:21.605 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.597+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280b3cb0 con 0x7f59280d0fc0 2024-09-21T07:26:21.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.598+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5924005030 con 0x7f59280d0fc0 2024-09-21T07:26:21.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.599+0000 7f591e7fc640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 0x7f590c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.599+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f592409b540 con 0x7f59280d0fc0 2024-09-21T07:26:21.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.599+0000 7f592d636640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 0x7f590c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.599+0000 7f592d636640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 0x7f590c07b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59280a18d0 tx=0x7f591800b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.603+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5924020020 con 0x7f59280d0fc0 2024-09-21T07:26:21.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.781+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59281e8290 con 0x7f59280d0fc0 2024-09-21T07:26:21.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.783+0000 7f591e7fc640 1 -- 172.21.15.105:0/2711536235 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18683 (secure 0 0 0) 0x7f5924017030 con 0x7f59280d0fc0 2024-09-21T07:26:21.785 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:21.785 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":26,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:20.139487+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 msgr2=0x7f590c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 0x7f590c07b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59280a18d0 tx=0x7f591800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 msgr2=0x7f59281f9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f592400d690 tx=0x7f592400db50 comp rx=0 tx=0).stop 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 shutdown_connections 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f590c078b90 0x7f590c07b0a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59280d3020 0x7f592815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59280d4460 0x7f59281f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 --2- 172.21.15.105:0/2711536235 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59280d0fc0 0x7f59281f9750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 >> 172.21.15.105:0/2711536235 conn(0x7f592808bfb0 msgr2=0x7f5928161c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 shutdown_connections 2024-09-21T07:26:21.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.785+0000 7f593009b640 1 -- 172.21.15.105:0/2711536235 wait complete. 2024-09-21T07:26:21.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:21.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 -- 172.21.15.105:0/1025504730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a800d0b90 msgr2=0x7f7a800b3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 --2- 172.21.15.105:0/1025504730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a800d0b90 0x7f7a800b3e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009830 tx=0x7f7a6802a510 comp rx=0 tx=0).stop 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 -- 172.21.15.105:0/1025504730 shutdown_connections 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 --2- 172.21.15.105:0/1025504730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a800d0b90 0x7f7a800b3e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 -- 172.21.15.105:0/1025504730 >> 172.21.15.105:0/1025504730 conn(0x7f7a800cee50 msgr2=0x7f7a800cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 -- 172.21.15.105:0/1025504730 shutdown_connections 2024-09-21T07:26:21.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.926+0000 7f7a86b69640 1 -- 172.21.15.105:0/1025504730 wait complete. 2024-09-21T07:26:21.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.928+0000 7f7a86b69640 1 Processor -- start 2024-09-21T07:26:21.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.928+0000 7f7a86b69640 1 -- start start 2024-09-21T07:26:21.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 0x7f7a801f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 0x7f7a80150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800c65a0 con 0x7f7a801609c0 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7a800d11b0 con 0x7f7a800b4390 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a86b69640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800c6720 con 0x7f7a800d0b90 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.929+0000 7f7a85106640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38542/0 (socket says 172.21.15.105:38542) 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 -- 172.21.15.105:0/2889881730 learned_addr learned my addr 172.21.15.105:0/2889881730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a84905640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 0x7f7a801f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a7ffff640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 0x7f7a80150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 -- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 msgr2=0x7f7a80150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 0x7f7a80150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 -- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 msgr2=0x7f7a801f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:21.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 0x7f7a801f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 -- 172.21.15.105:0/2889881730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800ca780 con 0x7f7a801609c0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a7ffff640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 0x7f7a80150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a84905640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 0x7f7a801f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a85106640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c730 tx=0x7f7a7800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78017070 con 0x7f7a801609c0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a78003c60 con 0x7f7a801609c0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a801f76f0 con 0x7f7a801609c0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a780052d0 con 0x7f7a801609c0 2024-09-21T07:26:21.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.930+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a801f7cb0 con 0x7f7a801609c0 2024-09-21T07:26:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.931+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a800b3e50 con 0x7f7a801609c0 2024-09-21T07:26:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.932+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a7801a070 con 0x7f7a801609c0 2024-09-21T07:26:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.932+0000 7f7a7d7fa640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 0x7f7a6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:21.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.933+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7a7809a8c0 con 0x7f7a801609c0 2024-09-21T07:26:21.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.933+0000 7f7a84905640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 0x7f7a6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:21.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.933+0000 7f7a84905640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 0x7f7a6007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a68004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:21.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:21.937+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a800b47a0 con 0x7f7a801609c0 2024-09-21T07:26:22.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.151+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a801f7450 con 0x7f7a801609c0 2024-09-21T07:26:22.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.152+0000 7f7a7d7fa640 1 -- 172.21.15.105:0/2889881730 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5231 (secure 0 0 0) 0x7f7a780629c0 con 0x7f7a801609c0 2024-09-21T07:26:22.154 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:22.154 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":7,"btime":"2024-09-21T07:26:20:784760+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:20.139726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:22.154 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 7 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 msgr2=0x7f7a6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 0x7f7a6007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7a68034000 tx=0x7f7a68004730 comp rx=0 tx=0).stop 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 msgr2=0x7f7a80158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c730 tx=0x7f7a7800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 shutdown_connections 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a60078b90 0x7f7a6007b0a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7a801609c0 0x7f7a80158a20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a800d0b90 0x7f7a80150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 --2- 172.21.15.105:0/2889881730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a800b4390 0x7f7a801f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 >> 172.21.15.105:0/2889881730 conn(0x7f7a800cee50 msgr2=0x7f7a800c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 shutdown_connections 2024-09-21T07:26:22.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.154+0000 7f7a86b69640 1 -- 172.21.15.105:0/2889881730 wait complete. 2024-09-21T07:26:22.167 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 -- 172.21.15.105:0/1159447928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394415f0f0 msgr2=0x7f394415f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 --2- 172.21.15.105:0/1159447928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394415f0f0 0x7f394415f500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3944074c00 tx=0x7f393c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 -- 172.21.15.105:0/1159447928 shutdown_connections 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 --2- 172.21.15.105:0/1159447928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f394415f0f0 0x7f394415f500 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 --2- 172.21.15.105:0/1159447928 >> v2:172.21.15.112:3300/0 conn(0x7f3944073ea0 0x7f39440742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 --2- 172.21.15.105:0/1159447928 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415c7f0 0x7f39440cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.278+0000 7f3949b62640 1 -- 172.21.15.105:0/1159447928 >> 172.21.15.105:0/1159447928 conn(0x7f39440d1270 msgr2=0x7f3944073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.279+0000 7f3949b62640 1 -- 172.21.15.105:0/1159447928 shutdown_connections 2024-09-21T07:26:22.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.279+0000 7f3949b62640 1 -- 172.21.15.105:0/1159447928 wait complete. 2024-09-21T07:26:22.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.280+0000 7f3949b62640 1 Processor -- start 2024-09-21T07:26:22.282 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.280+0000 7f3949b62640 1 -- start start 2024-09-21T07:26:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f394415c7f0 0x7f39441457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415f0f0 0x7f39441547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3944152010 con 0x7f3944073ea0 2024-09-21T07:26:22.285 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3944151e90 con 0x7f394415c7f0 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f3944152190 con 0x7f394415f0f0 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38562/0 (socket says 172.21.15.105:38562) 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 -- 172.21.15.105:0/3540664669 learned_addr learned my addr 172.21.15.105:0/3540664669 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3942ffd640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f394415c7f0 0x7f39441457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3943fff640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415f0f0 0x7f39441547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 -- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415f0f0 msgr2=0x7f39441547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415f0f0 0x7f39441547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 -- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f394415c7f0 msgr2=0x7f39441457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f394415c7f0 0x7f39441457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 -- 172.21.15.105:0/3540664669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39441f8990 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f39437fe640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f393400d6d0 tx=0x7f393400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934017070 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3934005330 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39441f8c70 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934009820 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.282+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39441f9230 con 0x7f3944073ea0 2024-09-21T07:26:22.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.283+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39440cd160 con 0x7f3944073ea0 2024-09-21T07:26:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.284+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39340099c0 con 0x7f3944073ea0 2024-09-21T07:26:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.285+0000 7f392bfff640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 0x7f393007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.285+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f393409afb0 con 0x7f3944073ea0 2024-09-21T07:26:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.285+0000 7f3942ffd640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 0x7f393007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:22.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.285+0000 7f3942ffd640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 0x7f393007b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3944155df0 tx=0x7f393800b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:22.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.289+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3934062af0 con 0x7f3944073ea0 2024-09-21T07:26:22.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.466+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39441f9610 con 0x7f3944073ea0 2024-09-21T07:26:22.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.467+0000 7f392bfff640 1 -- 172.21.15.105:0/3540664669 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f3934067520 con 0x7f3944073ea0 2024-09-21T07:26:22.469 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:22.469 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:22.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 msgr2=0x7f393007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 0x7f393007b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3944155df0 tx=0x7f393800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 msgr2=0x7f3944146a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f393400d6d0 tx=0x7f393400db90 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 shutdown_connections 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3930078bc0 0x7f393007b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f394415c7f0 0x7f39441457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f394415f0f0 0x7f39441547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 --2- 172.21.15.105:0/3540664669 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3944073ea0 0x7f3944146a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 >> 172.21.15.105:0/3540664669 conn(0x7f39440d1270 msgr2=0x7f39441514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 shutdown_connections 2024-09-21T07:26:22.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.469+0000 7f3949b62640 1 -- 172.21.15.105:0/3540664669 wait complete. 2024-09-21T07:26:22.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:26:15.095198+0000', 'modified': '2024-09-21T07:26:20.139726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4434': {'gid': 4434, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6836/3289542241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3289542241}, {'type': 'v1', 'addr': '172.21.15.112:6836', 'nonce': 3289542241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4434, 'qdb_cluster': [4434]} 2024-09-21T07:26:22.886 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:26:22.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 -- 172.21.15.105:0/546372824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 msgr2=0x7fafbc06f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/546372824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc06f040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0d2140 tx=0x7fafac02a910 comp rx=0 tx=0).stop 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 -- 172.21.15.105:0/546372824 shutdown_connections 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/546372824 >> v2:172.21.15.112:3300/0 conn(0x7fafbc0b9790 0x7fafbc0d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/546372824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc06f040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 -- 172.21.15.105:0/546372824 >> 172.21.15.105:0/546372824 conn(0x7fafbc09d850 msgr2=0x7fafbc08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 -- 172.21.15.105:0/546372824 shutdown_connections 2024-09-21T07:26:23.001 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.998+0000 7fafc3c7e640 1 -- 172.21.15.105:0/546372824 wait complete. 2024-09-21T07:26:23.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:22.999+0000 7fafc3c7e640 1 Processor -- start 2024-09-21T07:26:23.002 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.000+0000 7fafc3c7e640 1 -- start start 2024-09-21T07:26:23.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 0x7fafbc1517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 0x7fafbc0b8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc0d0e70 con 0x7fafbc0b3710 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc0d0cf0 con 0x7fafbc0d3880 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc3c7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc0d0ff0 con 0x7fafbc0b9790 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38582/0 (socket says 172.21.15.105:38582) 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 -- 172.21.15.105:0/2887632824 learned_addr learned my addr 172.21.15.105:0/2887632824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1219640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 0x7fafbc1517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc221b640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 0x7fafbc0b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 -- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 msgr2=0x7fafbc1517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 0x7fafbc1517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 -- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 msgr2=0x7fafbc0b8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 0x7fafbc0b8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 -- 172.21.15.105:0/2887632824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1f97c0 con 0x7fafbc0b3710 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1219640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 0x7fafbc1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc221b640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 0x7fafbc0b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafac033040 tx=0x7fafac003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafba7fc640 1 -- 172.21.15.105:0/2887632824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac004240 con 0x7fafbc0b3710 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.001+0000 7fafba7fc640 1 -- 172.21.15.105:0/2887632824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafac004400 con 0x7fafbc0b3710 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.002+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1f9aa0 con 0x7fafbc0b3710 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.002+0000 7fafba7fc640 1 -- 172.21.15.105:0/2887632824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac004850 con 0x7fafbc0b3710 2024-09-21T07:26:23.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.002+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc15f640 con 0x7fafbc0b3710 2024-09-21T07:26:23.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.003+0000 7fafba7fc640 1 -- 172.21.15.105:0/2887632824 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fafac00c040 con 0x7fafbc0b3710 2024-09-21T07:26:23.007 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.004+0000 7fafba7fc640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 0x7fafa807b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.004+0000 7fafba7fc640 1 -- 172.21.15.105:0/2887632824 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fafac0bc980 con 0x7fafbc0b3710 2024-09-21T07:26:23.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.004+0000 7fafc1219640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 0x7fafa807b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.005+0000 7fafcabe3740 1 Processor -- start 2024-09-21T07:26:23.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.005+0000 7fafc1219640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 0x7fafa807b240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0caba0 tx=0x7fafb0005f10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.008 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.005+0000 7fafcabe3740 1 -- start start 2024-09-21T07:26:23.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce6b270 0x558d6d23cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d23d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558d6d185820 con 0x558d6ce4a590 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d6d174c50 con 0x558d6ce6b270 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x558d6d178a60 con 0x558d6ce423b0 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38588/0 (socket says 172.21.15.105:38588) 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 -- 172.21.15.105:0/1712543985 learned_addr learned my addr 172.21.15.105:0/1712543985 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce6b270 0x558d6d23cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc1219640 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d23d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 -- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 msgr2=0x558d6d23d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d23d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 -- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce6b270 msgr2=0x558d6d23cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce6b270 0x558d6d23cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.006+0000 7fafc221b640 1 -- 172.21.15.105:0/1712543985 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6ced6850 con 0x558d6ce4a590 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafc221b640 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb400b7a0 tx=0x7fafb400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ce4398c23aa79a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafa37fe640 1 -- 172.21.15.105:0/1712543985 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4018020 con 0x558d6ce4a590 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafa37fe640 1 -- 172.21.15.105:0/1712543985 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb4004110 con 0x558d6ce4a590 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafa37fe640 1 -- 172.21.15.105:0/1712543985 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb400d740 con 0x558d6ce4a590 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafcabe3740 1 -- 172.21.15.105:0/1712543985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 msgr2=0x558d6d25a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.007+0000 7fafcabe3740 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb400b7a0 tx=0x7fafb400bba0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 -- 172.21.15.105:0/1712543985 shutdown_connections 2024-09-21T07:26:23.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d25a3e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d23d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 --2- 172.21.15.105:0/1712543985 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce6b270 0x558d6d23cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 -- 172.21.15.105:0/1712543985 >> 172.21.15.105:0/1712543985 conn(0x558d6ce5f3c0 msgr2=0x558d6d213260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 -- 172.21.15.105:0/1712543985 shutdown_connections 2024-09-21T07:26:23.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 -- 172.21.15.105:0/1712543985 wait complete. 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.008+0000 7fafcabe3740 1 -- start start 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.009+0000 7fafcabe3740 1 Processor -- start 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce4a590 0x558d6d24fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce6b270 0x558d6d267630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558d6cee5620 con 0x558d6ce423b0 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d6d11a5f0 con 0x558d6ce4a590 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x558d6d23da20 con 0x558d6ce6b270 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafcabe3740 10 client.0 _ll_register_callbacks cb 0x558d6d11e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38590/0 (socket says 172.21.15.105:38590) 2024-09-21T07:26:23.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 -- 172.21.15.105:0/531357490 learned_addr learned my addr 172.21.15.105:0/531357490 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1219640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce4a590 0x558d6d24fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc221b640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce6b270 0x558d6d267630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 -- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce6b270 msgr2=0x558d6d267630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce6b270 0x558d6d267630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 -- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce4a590 msgr2=0x558d6d24fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce4a590 0x558d6d24fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 -- 172.21.15.105:0/531357490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6cf4d6b0 con 0x558d6ce423b0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafac000f80 tx=0x7fafac02a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784babbd7ec360ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.010+0000 7fafa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac08db10 con 0x558d6ce423b0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fafac0a6b80 con 0x558d6ce423b0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb4004110 con 0x558d6ce423b0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fafac084b10 con 0x558d6ce423b0 2024-09-21T07:26:23.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 10 client.4551 fetch_fsmap learned FSMap version 7 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558d6d250740 con 0x558d6ce423b0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb4004110 con 0x558d6ce423b0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7fafac083670 con 0x558d6ce423b0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 10 client.4551 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 10 client.4551 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 20 client.4551 populate_metadata read hostname 'smithi105' 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6ce29e40 0x558d6d24edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 4 client.4551 mds_command: new command op to 4434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 --> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d6d187770 con 0x558d6ce29e40 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.011+0000 7fafc1219640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6ce29e40 0x558d6d24edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.012+0000 7fafa0ff9640 10 client.4551 ms_handle_connect on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.012+0000 7fafc1219640 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6ce29e40 0x558d6d24edc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195b0203e656fcd1 server_cookie=23bbdca7763826df in_seq=0 out_seq=0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafa0ff9640 1 -- 172.21.15.105:0/531357490 <== mds.0 v2:172.21.15.112:6834/3289542241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d6d187770 con 0x558d6ce29e40 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafa0ff9640 10 client.4551 handle_command_reply: tid=0 2024-09-21T07:26:23.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 client.4551 shutdown 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 msgr2=0x558d6d26aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafac000f80 tx=0x7fafac02a910 comp rx=0 tx=0).stop 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 shutdown_connections 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6ce29e40 0x558d6d24edc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce4a590 0x558d6d24fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafa0ff9640 0 client.4551 ms_handle_reset on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce6b270 0x558d6d267630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 --2- 172.21.15.105:0/531357490 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d26aee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 >> 172.21.15.105:0/531357490 conn(0x558d6ce5f3c0 msgr2=0x558d6d213110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 shutdown_connections 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 1 -- 172.21.15.105:0/531357490 wait complete. 2024-09-21T07:26:23.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.013+0000 7fafcabe3740 20 client.4551 trim_cache size 0 max 16384 2024-09-21T07:26:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.031+0000 7fafcabe3740 1 Processor -- start 2024-09-21T07:26:23.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.031+0000 7fafcabe3740 1 -- start start 2024-09-21T07:26:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d22a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.036 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce4a590 0x558d6d2130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc2027e0 con 0x558d6ce423b0 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc201c10 con 0x558d6ce29e40 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc202190 con 0x558d6ce4a590 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafc1219640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafc1219640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38606/0 (socket says 172.21.15.105:38606) 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.032+0000 7fafc1219640 1 -- 172.21.15.105:0/2220617861 learned_addr learned my addr 172.21.15.105:0/2220617861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 -- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce4a590 msgr2=0x558d6d2130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce4a590 0x558d6d2130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 -- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 msgr2=0x558d6d22a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.037 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d22a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d22a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 -- 172.21.15.105:0/2220617861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6d267990 con 0x558d6ce423b0 2024-09-21T07:26:23.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafc1219640 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b3eb0 tx=0x7fafb000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9b2d1cb6f1a3e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.041 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafa2ffd640 1 -- 172.21.15.105:0/2220617861 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0005020 con 0x558d6ce423b0 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafa2ffd640 1 -- 172.21.15.105:0/2220617861 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb00051c0 con 0x558d6ce423b0 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.033+0000 7fafa2ffd640 1 -- 172.21.15.105:0/2220617861 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb00055d0 con 0x558d6ce423b0 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 -- 172.21.15.105:0/2220617861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 msgr2=0x558d6d22af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b3eb0 tx=0x7fafb000bf90 comp rx=0 tx=0).stop 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 -- 172.21.15.105:0/2220617861 shutdown_connections 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce4a590 0x558d6d2130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce423b0 0x558d6d22af10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 --2- 172.21.15.105:0/2220617861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d22a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 -- 172.21.15.105:0/2220617861 >> 172.21.15.105:0/2220617861 conn(0x558d6ce5f3c0 msgr2=0x558d6d20db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 -- 172.21.15.105:0/2220617861 shutdown_connections 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.034+0000 7fafcabe3740 1 -- 172.21.15.105:0/2220617861 wait complete. 2024-09-21T07:26:23.042 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.035+0000 7fafcabe3740 1 -- start start 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.035+0000 7fafcabe3740 1 Processor -- start 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d2d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d22a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x558d6d2524b0 con 0x558d6ce4a590 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558d6d267c20 con 0x558d6ce29e40 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x558d6d2284e0 con 0x558d6ce423b0 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafcabe3740 10 client.0 _ll_register_callbacks cb 0x7fafac080b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38622/0 (socket says 172.21.15.105:38622) 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 -- 172.21.15.105:0/3912156581 learned_addr learned my addr 172.21.15.105:0/3912156581 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d2d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.043 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc1219640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d22a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 -- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 msgr2=0x558d6d22a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d22a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 -- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 msgr2=0x558d6d2d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d2d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc221b640 1 -- 172.21.15.105:0/3912156581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6d2148f0 con 0x558d6ce4a590 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d2d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.036+0000 7fafc1219640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d22a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafc221b640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb40043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bdb7ad7dff9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.044 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:26:23.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4021020 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fafb400db50 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb00051c0 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fafb402b050 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 10 client.4557 fetch_fsmap learned FSMap version 7 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x558d6d27b390 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafb00051c0 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7fafb402e020 con 0x558d6ce4a590 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 10 client.4557 fetch_fsmap finished waiting for FSMap version 7 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 10 client.4557 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 20 client.4557 populate_metadata read hostname 'smithi105' 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6cdff280 0x558d6d23cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 4 client.4557 mds_command: new command op to 4434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:26:23.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 --> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] -- command(tid 0: {"prefix": "status"}) -- 0x558d6d253f70 con 0x558d6cdff280 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.037+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6cdff280 0x558d6d23cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafa0ff9640 10 client.4557 ms_handle_connect on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafc1a1a640 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6cdff280 0x558d6d23cc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30aef5309a0b66c1 server_cookie=c16caf69873107e4 in_seq=0 out_seq=0 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafa0ff9640 1 -- 172.21.15.105:0/3912156581 <== mds.0 v2:172.21.15.112:6834/3289542241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x558d6d253f70 con 0x558d6cdff280 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafa0ff9640 10 client.4557 handle_command_reply: tid=0 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 client.4557 shutdown 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 msgr2=0x558d6d22af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fafb4006fd0 tx=0x7fafb40043d0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 shutdown_connections 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x558d6cdff280 0x558d6d23cc40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x558d6ce4a590 0x558d6d22af10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafa0ff9640 0 client.4557 ms_handle_reset on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x558d6ce423b0 0x558d6d22a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 --2- 172.21.15.105:0/3912156581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558d6ce29e40 0x558d6d2d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 >> 172.21.15.105:0/3912156581 conn(0x558d6ce5f3c0 msgr2=0x558d6d263bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 shutdown_connections 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 1 -- 172.21.15.105:0/3912156581 wait complete. 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.038+0000 7fafcabe3740 20 client.4557 trim_cache size 0 max 16384 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15ee7004-a3e2-41ce-b1eb-6358972a1ae2", 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4434, 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-21T07:26:23.049 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 7.2828858399999996, 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 7, 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 27, 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 26, 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 13.322432116 2024-09-21T07:26:23.050 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:26:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 msgr2=0x7fafa807b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 0x7fafa807b240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0caba0 tx=0x7fafb0005f10 comp rx=0 tx=0).stop 2024-09-21T07:26:23.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 msgr2=0x7fafbc1f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafac033040 tx=0x7fafac003c20 comp rx=0 tx=0).stop 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 shutdown_connections 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fafa8078d30 0x7fafa807b240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafbc0d3880 0x7fafbc0b8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fafbc0b9790 0x7fafbc1517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 --2- 172.21.15.105:0/2887632824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fafbc0b3710 0x7fafbc1f8f70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 >> 172.21.15.105:0/2887632824 conn(0x7fafbc09d850 msgr2=0x7fafbc0d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 shutdown_connections 2024-09-21T07:26:23.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.039+0000 7fafc3c7e640 1 -- 172.21.15.105:0/2887632824 wait complete. 2024-09-21T07:26:23.052 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:26:23.053 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:23.053 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:26:23.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 -- 172.21.15.105:0/2366040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf80b6440 msgr2=0x7fecf80cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 --2- 172.21.15.105:0/2366040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf80b6440 0x7fecf80cf050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fecf806ff60 tx=0x7fece802a540 comp rx=0 tx=0).stop 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 -- 172.21.15.105:0/2366040370 shutdown_connections 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 --2- 172.21.15.105:0/2366040370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6850 0x7fecf80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 --2- 172.21.15.105:0/2366040370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf80b6440 0x7fecf80cf050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 --2- 172.21.15.105:0/2366040370 >> v2:172.21.15.112:3300/0 conn(0x7fecf808c3c0 0x7fecf806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 -- 172.21.15.105:0/2366040370 >> 172.21.15.105:0/2366040370 conn(0x7fecf808bfb0 msgr2=0x7fecf80d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 -- 172.21.15.105:0/2366040370 shutdown_connections 2024-09-21T07:26:23.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.166+0000 7fecff8f9640 1 -- 172.21.15.105:0/2366040370 wait complete. 2024-09-21T07:26:23.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.168+0000 7fecff8f9640 1 Processor -- start 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.168+0000 7fecff8f9640 1 -- start start 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 0x7fecf81f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf80b6850 0x7fecf8160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80cfad0 con 0x7fecf808c3c0 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fecf80b3b60 con 0x7fecf80b6850 2024-09-21T07:26:23.175 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecff8f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fecf80cfc50 con 0x7fecf80b6440 2024-09-21T07:26:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38630/0 (socket says 172.21.15.105:38630) 2024-09-21T07:26:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 -- 172.21.15.105:0/1789905423 learned_addr learned my addr 172.21.15.105:0/1789905423 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfce94640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 0x7fecf81f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfde96640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf80b6850 0x7fecf8160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 -- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 msgr2=0x7fecf81f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 0x7fecf81f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 -- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf80b6850 msgr2=0x7fecf8160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf80b6850 0x7fecf8160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 -- 172.21.15.105:0/1789905423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf8160570 con 0x7fecf808c3c0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfce94640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 0x7fecf81f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.169+0000 7fecfd695640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecf400c940 tx=0x7fecf400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.170+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004220 con 0x7fecf808c3c0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.170+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecf40079b0 con 0x7fecf808c3c0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.170+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf81f6360 con 0x7fecf808c3c0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.170+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4007e20 con 0x7fecf808c3c0 2024-09-21T07:26:23.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.170+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf815f830 con 0x7fecf808c3c0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.171+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf806f7f0 con 0x7fecf808c3c0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.171+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fecf4005020 con 0x7fecf808c3c0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.172+0000 7fecedffb640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 0x7fecdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.172+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fecf409b400 con 0x7fecf808c3c0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.172+0000 7fecfce94640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 0x7fecdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.172+0000 7fecfce94640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 0x7fecdc07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:23.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.176+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fecf4014030 con 0x7fecf808c3c0 2024-09-21T07:26:23.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.399+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fecf80b20b0 con 0x7fecf808c3c0 2024-09-21T07:26:23.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.399+0000 7fecedffb640 1 -- 172.21.15.105:0/1789905423 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fecf4063590 con 0x7fecf808c3c0 2024-09-21T07:26:23.402 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 msgr2=0x7fecdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 0x7fecdc07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fece8033000 tx=0x7fece8004730 comp rx=0 tx=0).stop 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 msgr2=0x7fecf8160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecf400c940 tx=0x7fecf400ce00 comp rx=0 tx=0).stop 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 shutdown_connections 2024-09-21T07:26:23.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fecdc078b90 0x7fecdc07b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fecf80b6850 0x7fecf8160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fecf80b6440 0x7fecf81f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 --2- 172.21.15.105:0/1789905423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fecf808c3c0 0x7fecf8160030 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 >> 172.21.15.105:0/1789905423 conn(0x7fecf808bfb0 msgr2=0x7fecf80d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 shutdown_connections 2024-09-21T07:26:23.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23.402+0000 7fecff8f9640 1 -- 172.21.15.105:0/1789905423 wait complete. 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:26:23.415 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T07:26:23.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:23.416 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-09-21T07:26:23.432 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:23.432 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:23.433 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:26:23.434 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:23.434 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:26:23.434 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42790sec preferred_lft 42790sec 2024-09-21T07:26:23.434 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-21T07:26:23.434 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:23.434 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:26:23.434 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:26:23.435 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:23.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:23.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:23.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:23.633 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:26:23.662 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-09-21T07:26:23.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:23.664 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-09-21T07:26:23.738 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-09-21T07:26:23.738 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:26:23.738 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:26:23.738 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:26:23.738 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:26:23.740 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:23.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:24.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:24.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:24.061 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:26:24.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:24.084 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-09-21T07:26:24.139 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:24.139 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:24.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:24.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-21T07:26:24.139 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:24.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:24.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:24.255 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-21T07:26:24.256 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:24.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:24.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:24.380 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:24.381 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:24.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-09-21T07:26:24.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:26:24.381 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:24.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:24.451 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:24.456 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-21T07:26:24.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:24.457 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:24.472 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-21T07:26:24.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:24.472 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:24.526 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-21T07:26:24.592 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-21T07:26:24.647 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:26:24.648 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:24.649 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:24.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:24.650 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-21T07:26:24.706 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-21T07:26:24.706 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-21T07:26:24.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-21T07:26:24.774 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-21T07:26:24.830 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:26:24.831 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:24.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:24.832 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-21T07:26:24.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:2024-09-21T07:26:24.952+0000 7fe1d658a480 -1 init, newargv = 0x559e62180910 newargc=15 2024-09-21T07:26:24.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44041]: starting ceph client 2024-09-21T07:26:24.961 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44041]: starting fuse 2024-09-21T07:26:25.988 DEBUG:teuthology.orchestra.run.smithi105:> sudo modprobe fuse 2024-09-21T07:26:26.019 DEBUG:teuthology.orchestra.run.smithi105:> cat /proc/self/mounts | awk '{print $2}' 2024-09-21T07:26:26.079 INFO:teuthology.orchestra.run.smithi105.stdout:/proc 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/security 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/shm 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/pts 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/run 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/cgroup 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/pstore 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/bpf 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/ 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/selinux 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/proc/sys/fs/binfmt_misc 2024-09-21T07:26:26.080 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/mqueue 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/dev/hugepages 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/debug 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/tracing 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/fs/fuse/connections 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/sys/kernel/config 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-sysctl.service 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/nfs/rpc_pipefs 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/run/user/1000 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-0 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-1 2024-09-21T07:26:26.081 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-2 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run.smithi105.stdout:/var/lib/ceph/osd/ceph-3 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run.smithi105.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run.smithi105.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:26.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:26.082 DEBUG:teuthology.orchestra.run.smithi105:> ls /sys/fs/fuse/connections 2024-09-21T07:26:26.138 INFO:teuthology.orchestra.run.smithi105.stdout:41 2024-09-21T07:26:26.138 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> sudo stdin-killer -- python3 -c ' 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> import glob 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> import re 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> import os 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> import subprocess 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> def _find_admin_socket(client_name): 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> files = glob.glob(asok_path) 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-21T07:26:26.139 DEBUG:teuthology.orchestra.run.smithi105:> # Given a non-glob path, it better be there 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> if "*" not in asok_path: 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> assert(len(files) == 1) 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> return files[0] 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> for f in files: 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> contents = proc_f.read() 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> if mountpoint in contents: 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> return f 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> print(_find_admin_socket("client.admin")) 2024-09-21T07:26:26.140 DEBUG:teuthology.orchestra.run.smithi105:> ' 2024-09-21T07:26:26.264 INFO:teuthology.orchestra.run.smithi105.stdout:/var/run/ceph/ceph-client.admin.44041.asok 2024-09-21T07:26:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:26.276 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.44041.asok 2024-09-21T07:26:26.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:26.276 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.44041.asok status 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "metadata": { 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "entity_id": "admin", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "hostname": "smithi105", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "pid": "44041", 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "root": "/" 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:26:26.419 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_count": 0, 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "dentry_pinned_count": 0, 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4569, 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "inst": { 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "name": { 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "client", 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "num": 4569 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 4274337291 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:26:26.420 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": { 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "type": "v1", 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr": "192.168.0.1:0", 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "nonce": 4274337291 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "inst_str": "client.4569 192.168.0.1:0/4274337291", 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "addr_str": "192.168.0.1:0/4274337291", 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "inode_count": 1, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_epoch": 7, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch": 27, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_epoch_barrier": 0, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "blocklisted": false, 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs" 2024-09-21T07:26:26.421 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:26:26.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:26.427 DEBUG:teuthology.orchestra.run.smithi105:> stat --file-system '--printf=%T 2024-09-21T07:26:26.428 DEBUG:teuthology.orchestra.run.smithi105:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:26.443 INFO:teuthology.orchestra.run.smithi105.stdout:fuseblk 2024-09-21T07:26:26.443 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:26.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:26.443 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:26.509 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-21T07:26:26.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:26.616 INFO:teuthology.orchestra.run.smithi105.stdout:# file: . 2024-09-21T07:26:26.616 INFO:teuthology.orchestra.run.smithi105.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-21T07:26:26.616 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:26.616 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:26.622 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:26.622 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:26.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:26.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 -- 172.21.15.105:0/3132017383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c2c004080 msgr2=0x7f5c2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 --2- 172.21.15.105:0/3132017383 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c2c004080 0x7f5c2c004800 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5c400cfc60 tx=0x7f5c2c005830 comp rx=0 tx=0).stop 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c3f7fe640 1 -- 172.21.15.105:0/3132017383 reap_dead start 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 -- 172.21.15.105:0/3132017383 shutdown_connections 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 -- 172.21.15.105:0/3132017383 >> 172.21.15.105:0/3132017383 conn(0x7f5c400ccb90 msgr2=0x7f5c400ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 -- 172.21.15.105:0/3132017383 shutdown_connections 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.790+0000 7f5c45532640 1 -- 172.21.15.105:0/3132017383 wait complete. 2024-09-21T07:26:26.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.792+0000 7f5c45532640 1 Processor -- start 2024-09-21T07:26:26.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.792+0000 7f5c45532640 1 -- start start 2024-09-21T07:26:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2c004080 0x7f5c4009b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c4015b7a0 0x7f5c4015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:26.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400ce5e0 con 0x7f5c40156270 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5c400ce460 con 0x7f5c2c004080 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.793+0000 7f5c45532640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5c400ce760 con 0x7f5c4015b7a0 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38654/0 (socket says 172.21.15.105:38654) 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 -- 172.21.15.105:0/1578176970 learned_addr learned my addr 172.21.15.105:0/1578176970 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c367fc640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c4015b7a0 0x7f5c4015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3effd640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2c004080 0x7f5c4009b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 -- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c4015b7a0 msgr2=0x7f5c4015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c4015b7a0 0x7f5c4015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 -- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2c004080 msgr2=0x7f5c4009b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2c004080 0x7f5c4009b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 -- 172.21.15.105:0/1578176970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c40156bc0 con 0x7f5c40156270 2024-09-21T07:26:26.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c3f7fe640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5c400cabd0 tx=0x7f5c38004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38004b20 con 0x7f5c40156270 2024-09-21T07:26:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c38004cc0 con 0x7f5c40156270 2024-09-21T07:26:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4009b970 con 0x7f5c40156270 2024-09-21T07:26:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38010bb0 con 0x7f5c40156270 2024-09-21T07:26:26.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.794+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c400b40c0 con 0x7f5c40156270 2024-09-21T07:26:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.795+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c400cdf20 con 0x7f5c40156270 2024-09-21T07:26:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.796+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c3801c020 con 0x7f5c40156270 2024-09-21T07:26:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.797+0000 7f5c37fff640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 0x7f5c2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:26.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.797+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5c3809b500 con 0x7f5c40156270 2024-09-21T07:26:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.797+0000 7f5c3effd640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 0x7f5c2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:26.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.797+0000 7f5c3effd640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 0x7f5c2007b0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003700 tx=0x7f5c3000b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:26.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.801+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c40156bc0 con 0x7f5c40156270 2024-09-21T07:26:26.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.992+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5c400cd370 con 0x7f5c40156270 2024-09-21T07:26:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.992+0000 7f5c37fff640 1 -- 172.21.15.105:0/1578176970 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5c38063040 con 0x7f5c40156270 2024-09-21T07:26:26.994 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 msgr2=0x7f5c2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 0x7f5c2007b0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5c30003700 tx=0x7f5c3000b260 comp rx=0 tx=0).stop 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 msgr2=0x7f5c40156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5c400cabd0 tx=0x7f5c38004700 comp rx=0 tx=0).stop 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 shutdown_connections 2024-09-21T07:26:26.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5c20078bc0 0x7f5c2007b0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c40156270 0x7f5c40156680 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5c4015b7a0 0x7f5c4015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 --2- 172.21.15.105:0/1578176970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5c2c004080 0x7f5c4009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 >> 172.21.15.105:0/1578176970 conn(0x7f5c400ccb90 msgr2=0x7f5c400b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 shutdown_connections 2024-09-21T07:26:26.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:26.994+0000 7f5c45532640 1 -- 172.21.15.105:0/1578176970 wait complete. 2024-09-21T07:26:27.007 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:27.007 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:27.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 -- 172.21.15.105:0/670484900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 msgr2=0x7f2d7006f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 --2- 172.21.15.105:0/670484900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d7006f7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c005fd0 tx=0x7f2d6c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 -- 172.21.15.105:0/670484900 shutdown_connections 2024-09-21T07:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 --2- 172.21.15.105:0/670484900 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 0x7f2d700cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 --2- 172.21.15.105:0/670484900 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d7006f7f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 -- 172.21.15.105:0/670484900 >> 172.21.15.105:0/670484900 conn(0x7f2d7008bfb0 msgr2=0x7f2d700d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 -- 172.21.15.105:0/670484900 shutdown_connections 2024-09-21T07:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.176+0000 7f2d782d3640 1 -- 172.21.15.105:0/670484900 wait complete. 2024-09-21T07:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.178+0000 7f2d782d3640 1 Processor -- start 2024-09-21T07:26:27.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.178+0000 7f2d782d3640 1 -- start start 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d70161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 0x7f2d7015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700b3ad0 con 0x7f2d700b6440 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700b3950 con 0x7f2d7008c3c0 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d782d3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f2d70162c70 con 0x7f2d700b6850 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d7586e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d7586e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38670/0 (socket says 172.21.15.105:38670) 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d7586e640 1 -- 172.21.15.105:0/2819495976 learned_addr learned my addr 172.21.15.105:0/2819495976 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d7606f640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d70161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.179+0000 7f2d76870640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 0x7f2d7015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 -- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 msgr2=0x7f2d7015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 0x7f2d7015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 -- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 msgr2=0x7f2d70161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d70161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 -- 172.21.15.105:0/2819495976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700cb4c0 con 0x7f2d700b6440 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d7586e640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c340 tx=0x7f2d6000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60011790 con 0x7f2d700b6440 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d60017070 con 0x7f2d700b6440 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d70162810 con 0x7f2d700b6440 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60011cc0 con 0x7f2d700b6440 2024-09-21T07:26:27.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.180+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d70160c80 con 0x7f2d700b6440 2024-09-21T07:26:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.181+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d7006f7f0 con 0x7f2d700b6440 2024-09-21T07:26:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.182+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2d6001c460 con 0x7f2d700b6440 2024-09-21T07:26:27.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.182+0000 7f2d66ffd640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 0x7f2d5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.182+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2d6009aca0 con 0x7f2d700b6440 2024-09-21T07:26:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.182+0000 7f2d7606f640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 0x7f2d5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:27.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.183+0000 7f2d7606f640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 0x7f2d5407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:27.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.186+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d700cfca0 con 0x7f2d700b6440 2024-09-21T07:26:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.377+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d701625a0 con 0x7f2d700b6440 2024-09-21T07:26:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.377+0000 7f2d66ffd640 1 -- 172.21.15.105:0/2819495976 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f2d60062760 con 0x7f2d700b6440 2024-09-21T07:26:27.379 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 msgr2=0x7f2d5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 0x7f2d5407b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c034000 tx=0x7f2d6c004ab0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 msgr2=0x7f2d70161ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000c340 tx=0x7f2d6000c800 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 shutdown_connections 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2d54078b90 0x7f2d5407b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d700b6850 0x7f2d7015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2d700b6440 0x7f2d70161ca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 --2- 172.21.15.105:0/2819495976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d7008c3c0 0x7f2d70161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 >> 172.21.15.105:0/2819495976 conn(0x7f2d7008bfb0 msgr2=0x7f2d700b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 shutdown_connections 2024-09-21T07:26:27.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:27.380+0000 7f2d782d3640 1 -- 172.21.15.105:0/2819495976 wait complete. 2024-09-21T07:26:27.392 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi105.front.sepia.ceph.com... 2024-09-21T07:26:27.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:27.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-21T07:26:27.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi105.stderr:ceph-fuse[44041]: fuse finished with error 0 and tester_r 0 2024-09-21T07:26:27.417 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:26:33.519 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:26:33.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:33.581 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-21T07:26:33.622 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 -- 172.21.15.105:0/2134982320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a40004d40 msgr2=0x7f3a40005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2134982320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a40004d40 0x7f3a40005210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a500013d0 tx=0x7f3a40005830 comp rx=0 tx=0).stop 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5d8a4640 1 -- 172.21.15.105:0/2134982320 reap_dead start 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 -- 172.21.15.105:0/2134982320 shutdown_connections 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 -- 172.21.15.105:0/2134982320 >> 172.21.15.105:0/2134982320 conn(0x7f3a5809aa20 msgr2=0x7f3a580b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:33.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 -- 172.21.15.105:0/2134982320 shutdown_connections 2024-09-21T07:26:33.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.787+0000 7f3a5f307640 1 -- 172.21.15.105:0/2134982320 wait complete. 2024-09-21T07:26:33.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.789+0000 7f3a5f307640 1 Processor -- start 2024-09-21T07:26:33.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.789+0000 7f3a5f307640 1 -- start start 2024-09-21T07:26:33.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 0x7f3a581f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:33.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 0x7f3a5806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580b4f60 con 0x7f3a58153650 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a580b4de0 con 0x7f3a581f3c90 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5f307640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f3a580b50e0 con 0x7f3a40004d40 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59562/0 (socket says 172.21.15.105:59562) 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 -- 172.21.15.105:0/2733937569 learned_addr learned my addr 172.21.15.105:0/2733937569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d0a3640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 0x7f3a581f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5c8a2640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 0x7f3a5806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 -- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 msgr2=0x7f3a581f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:33.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 0x7f3a581f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 -- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 msgr2=0x7f3a5806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 0x7f3a5806d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d8a4640 1 -- 172.21.15.105:0/2733937569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a581f7250 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5d0a3640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 0x7f3a581f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.790+0000 7f3a5c8a2640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 0x7f3a5806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a5d8a4640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3a58072a20 tx=0x7f3a50003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50039070 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a5002ba40 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a581f7530 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5002be30 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.791+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a581f3770 con 0x7f3a58153650 2024-09-21T07:26:33.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.792+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a58154220 con 0x7f3a58153650 2024-09-21T07:26:33.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.793+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a50003d10 con 0x7f3a58153650 2024-09-21T07:26:33.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.793+0000 7f3a55ffb640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 0x7f3a4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:33.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.793+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3a500baf90 con 0x7f3a58153650 2024-09-21T07:26:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.794+0000 7f3a5d0a3640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 0x7f3a4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:33.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.794+0000 7f3a5d0a3640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 0x7f3a4407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a48006b70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:33.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.798+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a50083090 con 0x7f3a58153650 2024-09-21T07:26:33.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.977+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a581f6fe0 con 0x7f3a58153650 2024-09-21T07:26:33.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.978+0000 7f3a55ffb640 1 -- 172.21.15.105:0/2733937569 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f3a50082990 con 0x7f3a58153650 2024-09-21T07:26:33.980 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:33.980 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 msgr2=0x7f3a4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 0x7f3a4407b0d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a48006b70 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 msgr2=0x7f3a58151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3a58072a20 tx=0x7f3a50003ba0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 shutdown_connections 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3a44078bc0 0x7f3a4407b0d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a581f3c90 0x7f3a5806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a58153650 0x7f3a58151620 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 --2- 172.21.15.105:0/2733937569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3a40004d40 0x7f3a581f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.980+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 >> 172.21.15.105:0/2733937569 conn(0x7f3a5809aa20 msgr2=0x7f3a580b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.981+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 shutdown_connections 2024-09-21T07:26:33.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:33.981+0000 7f3a5f307640 1 -- 172.21.15.105:0/2733937569 wait complete. 2024-09-21T07:26:33.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 -- 172.21.15.105:0/727147961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000b2620 msgr2=0x7fdf00168080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 --2- 172.21.15.105:0/727147961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000b2620 0x7fdf00168080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdef8005fd0 tx=0x7fdef802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 -- 172.21.15.105:0/727147961 shutdown_connections 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 --2- 172.21.15.105:0/727147961 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000b2620 0x7fdf00168080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 -- 172.21.15.105:0/727147961 >> 172.21.15.105:0/727147961 conn(0x7fdf000cee20 msgr2=0x7fdf000cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 -- 172.21.15.105:0/727147961 shutdown_connections 2024-09-21T07:26:34.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.104+0000 7fdf06539640 1 -- 172.21.15.105:0/727147961 wait complete. 2024-09-21T07:26:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.105+0000 7fdf06539640 1 Processor -- start 2024-09-21T07:26:34.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.106+0000 7fdf06539640 1 -- start start 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000b1b00 0x7fdf000aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf000b2620 0x7fdf001e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0009bbb0 con 0x7fdf000cff70 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0009ba30 con 0x7fdf000b1b00 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf06539640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fdf000b2a30 con 0x7fdf000b2620 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59572/0 (socket says 172.21.15.105:59572) 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 -- 172.21.15.105:0/657832857 learned_addr learned my addr 172.21.15.105:0/657832857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdeff7fe640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf000b2620 0x7fdf001e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdefffff640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000b1b00 0x7fdf000aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 -- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf000b2620 msgr2=0x7fdf001e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf000b2620 0x7fdf001e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 -- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000b1b00 msgr2=0x7fdf000aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000b1b00 0x7fdf000aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.107+0000 7fdf04ad6640 1 -- 172.21.15.105:0/657832857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf001e0960 con 0x7fdf000cff70 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdf04ad6640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdef8006010 tx=0x7fdef8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8031b90 con 0x7fdf000cff70 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdef8031e20 con 0x7fdf000cff70 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf001e0c40 con 0x7fdf000cff70 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8033310 con 0x7fdf000cff70 2024-09-21T07:26:34.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.108+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf001e1200 con 0x7fdf000cff70 2024-09-21T07:26:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.109+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf000b1f10 con 0x7fdf000cff70 2024-09-21T07:26:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.109+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdef8031660 con 0x7fdf000cff70 2024-09-21T07:26:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.110+0000 7fdefcff9640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 0x7fdeec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.110+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdef80bc5f0 con 0x7fdf000cff70 2024-09-21T07:26:34.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.110+0000 7fdefffff640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 0x7fdeec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.111+0000 7fdefffff640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 0x7fdeec07b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000f60 tx=0x7fdef0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.114+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdef803f030 con 0x7fdf000cff70 2024-09-21T07:26:34.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.292+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf000b81d0 con 0x7fdf000cff70 2024-09-21T07:26:34.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.293+0000 7fdefcff9640 1 -- 172.21.15.105:0/657832857 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7fdef8084780 con 0x7fdf000cff70 2024-09-21T07:26:34.295 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:34.296 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 msgr2=0x7fdeec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 0x7fdeec07b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdef0000f60 tx=0x7fdef0009040 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 msgr2=0x7fdf001e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdef8006010 tx=0x7fdef8003ba0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 shutdown_connections 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdeec078bc0 0x7fdeec07b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdf000cff70 0x7fdf001e2d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdf000b2620 0x7fdf001e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 --2- 172.21.15.105:0/657832857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf000b1b00 0x7fdf000aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.295+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 >> 172.21.15.105:0/657832857 conn(0x7fdf000cee20 msgr2=0x7fdf001e23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.296+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 shutdown_connections 2024-09-21T07:26:34.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.296+0000 7fdf06539640 1 -- 172.21.15.105:0/657832857 wait complete. 2024-09-21T07:26:34.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:26:34.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 -- 172.21.15.105:0/2530531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900d0310 msgr2=0x7f34900ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 --2- 172.21.15.105:0/2530531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900d0310 0x7f34900ccf40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f34900b74b0 tx=0x7f348802aa80 comp rx=0 tx=0).stop 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 -- 172.21.15.105:0/2530531 shutdown_connections 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 --2- 172.21.15.105:0/2530531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900d0310 0x7f34900ccf40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 --2- 172.21.15.105:0/2530531 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900b7940 0x7f34900b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 -- 172.21.15.105:0/2530531 >> 172.21.15.105:0/2530531 conn(0x7f34900cedb0 msgr2=0x7f34900cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 -- 172.21.15.105:0/2530531 shutdown_connections 2024-09-21T07:26:34.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.422+0000 7f349780e640 1 -- 172.21.15.105:0/2530531 wait complete. 2024-09-21T07:26:34.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.424+0000 7f349780e640 1 Processor -- start 2024-09-21T07:26:34.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.424+0000 7f349780e640 1 -- start start 2024-09-21T07:26:34.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34900cff00 0x7f34900c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900d0310 0x7f34900c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f34900b1d10 con 0x7f34900b7940 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34900cfce0 con 0x7f34900cff00 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f349780e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f34900b1e90 con 0x7f34900d0310 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.425+0000 7f34955aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59600/0 (socket says 172.21.15.105:59600) 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 -- 172.21.15.105:0/242891167 learned_addr learned my addr 172.21.15.105:0/242891167 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f3494da9640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34900cff00 0x7f34900c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f3495dab640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900d0310 0x7f34900c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 -- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900d0310 msgr2=0x7f34900c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900d0310 0x7f34900c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 -- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34900cff00 msgr2=0x7f34900c6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34900cff00 0x7f34900c6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 -- 172.21.15.105:0/242891167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34900cb3f0 con 0x7f34900b7940 2024-09-21T07:26:34.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f34955aa640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f348c00d700 tx=0x7f348c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c017070 con 0x7f34900b7940 2024-09-21T07:26:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f348c005310 con 0x7f34900b7940 2024-09-21T07:26:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34900cb6d0 con 0x7f34900b7940 2024-09-21T07:26:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f348c009820 con 0x7f34900b7940 2024-09-21T07:26:34.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.426+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34900cbc90 con 0x7f34900b7940 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.427+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34900b17d0 con 0x7f34900b7940 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.428+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f348c0099c0 con 0x7f34900b7940 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.428+0000 7f3485ffb640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.428+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f348c09b090 con 0x7f34900b7940 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.429+0000 7f3494da9640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.429+0000 7f3494da9640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 0x7f347407b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34901e5970 tx=0x7f3480009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.433+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f348c062b40 con 0x7f34900b7940 2024-09-21T07:26:34.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.647+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f34900b3c00 con 0x7f34900b7940 2024-09-21T07:26:34.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.826+0000 7f3485ffb640 1 -- 172.21.15.105:0/242891167 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f348c067570 con 0x7f34900b7940 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 msgr2=0x7f347407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 0x7f347407b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f34901e5970 tx=0x7f3480009040 comp rx=0 tx=0).stop 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 msgr2=0x7f34901e8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f348c00d700 tx=0x7f348c00dbc0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 shutdown_connections 2024-09-21T07:26:34.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3474078bc0 0x7f347407b0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34900cff00 0x7f34900c6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f34900d0310 0x7f34900c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 --2- 172.21.15.105:0/242891167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f34900b7940 0x7f34901e8230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 >> 172.21.15.105:0/242891167 conn(0x7f34900cedb0 msgr2=0x7f349015c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 shutdown_connections 2024-09-21T07:26:34.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.828+0000 7f349780e640 1 -- 172.21.15.105:0/242891167 wait complete. 2024-09-21T07:26:34.841 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-21T07:26:34.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.954+0000 7f682c8d3640 1 -- 172.21.15.105:0/23387598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280d0310 msgr2=0x7f68280ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.954+0000 7f682c8d3640 1 --2- 172.21.15.105:0/23387598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280d0310 0x7f68280ccf40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68280b74b0 tx=0x7f682002a530 comp rx=0 tx=0).stop 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 -- 172.21.15.105:0/23387598 shutdown_connections 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 --2- 172.21.15.105:0/23387598 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280d0310 0x7f68280ccf40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 --2- 172.21.15.105:0/23387598 >> v2:172.21.15.112:3300/0 conn(0x7f68280b7940 0x7f68280b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 -- 172.21.15.105:0/23387598 >> 172.21.15.105:0/23387598 conn(0x7f68280cedb0 msgr2=0x7f68280cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 -- 172.21.15.105:0/23387598 shutdown_connections 2024-09-21T07:26:34.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.955+0000 7f682c8d3640 1 -- 172.21.15.105:0/23387598 wait complete. 2024-09-21T07:26:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.956+0000 7f682c8d3640 1 Processor -- start 2024-09-21T07:26:34.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.957+0000 7f682c8d3640 1 -- start start 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 0x7f6828154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68280d0310 0x7f68280ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68280b1d10 con 0x7f68280b7940 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68280cfce0 con 0x7f68280cff00 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682c8d3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f68280b1e90 con 0x7f68280d0310 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59610/0 (socket says 172.21.15.105:59610) 2024-09-21T07:26:34.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 -- 172.21.15.105:0/2026544692 learned_addr learned my addr 172.21.15.105:0/2026544692 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f6826d9d640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68280d0310 0x7f68280ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f6825d9b640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 0x7f6828154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 -- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68280d0310 msgr2=0x7f68280ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68280d0310 0x7f68280ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 -- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 msgr2=0x7f6828154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 0x7f6828154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 -- 172.21.15.105:0/2026544692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682806e380 con 0x7f68280b7940 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f6825d9b640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 0x7f6828154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.958+0000 7f682659c640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f681800eb30 tx=0x7f681800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.959+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818010430 con 0x7f68280b7940 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.959+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68180105d0 con 0x7f68280b7940 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.959+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682806e660 con 0x7f68280b7940 2024-09-21T07:26:34.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.959+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818010a20 con 0x7f68280b7940 2024-09-21T07:26:34.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.959+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682806ec20 con 0x7f68280b7940 2024-09-21T07:26:34.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.960+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68280b17d0 con 0x7f68280b7940 2024-09-21T07:26:34.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.961+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6818010bc0 con 0x7f68280b7940 2024-09-21T07:26:34.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.961+0000 7f6816ffd640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 0x7f680807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:34.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.961+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f681809bde0 con 0x7f68280b7940 2024-09-21T07:26:34.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.961+0000 7f6825d9b640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 0x7f680807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:34.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.962+0000 7f6825d9b640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 0x7f680807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6828051180 tx=0x7f681c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:34.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:34.966+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6818063f70 con 0x7f68280b7940 2024-09-21T07:26:35.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.180+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f68280b3c00 con 0x7f68280b7940 2024-09-21T07:26:35.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.827+0000 7f6816ffd640 1 -- 172.21.15.105:0/2026544692 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f6818063870 con 0x7f68280b7940 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.829+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 msgr2=0x7f680807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.829+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 0x7f680807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6828051180 tx=0x7f681c009040 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.829+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 msgr2=0x7f68281f9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.829+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f681800eb30 tx=0x7f681800c2d0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 shutdown_connections 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6808078b90 0x7f680807b0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68280cff00 0x7f6828154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68280d0310 0x7f68280ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 --2- 172.21.15.105:0/2026544692 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68280b7940 0x7f68281f9a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 >> 172.21.15.105:0/2026544692 conn(0x7f68280cedb0 msgr2=0x7f68280b31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 shutdown_connections 2024-09-21T07:26:35.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.830+0000 7f682c8d3640 1 -- 172.21.15.105:0/2026544692 wait complete. 2024-09-21T07:26:35.843 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:26:35.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:35.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 -- 172.21.15.105:0/4113525774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40cd520 msgr2=0x7f4dd40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/4113525774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40cd520 0x7f4dd40cd930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc009830 tx=0x7f4dcc02a480 comp rx=0 tx=0).stop 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 -- 172.21.15.105:0/4113525774 shutdown_connections 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/4113525774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd40cd520 0x7f4dd40cd930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/4113525774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 0x7f4dd40ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 -- 172.21.15.105:0/4113525774 >> 172.21.15.105:0/4113525774 conn(0x7f4dd40cedb0 msgr2=0x7f4dd40cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 -- 172.21.15.105:0/4113525774 shutdown_connections 2024-09-21T07:26:35.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.954+0000 7f4ddb69b640 1 -- 172.21.15.105:0/4113525774 wait complete. 2024-09-21T07:26:35.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.956+0000 7f4ddb69b640 1 Processor -- start 2024-09-21T07:26:35.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.956+0000 7f4ddb69b640 1 -- start start 2024-09-21T07:26:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 0x7f4dd414b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd40cd520 0x7f4dd415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd40b3650 con 0x7f4dd409ba30 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd40b34d0 con 0x7f4dd40cd520 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4ddb69b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f4dd4167000 con 0x7f4dd40c59c0 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4dd9437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4dd9437640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59614/0 (socket says 172.21.15.105:59614) 2024-09-21T07:26:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.957+0000 7f4dd9437640 1 -- 172.21.15.105:0/2241885519 learned_addr learned my addr 172.21.15.105:0/2241885519 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd8c36640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 0x7f4dd414b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 -- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 msgr2=0x7f4dd414b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9c38640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd40cd520 0x7f4dd415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 0x7f4dd414b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 -- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd40cd520 msgr2=0x7f4dd415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd40cd520 0x7f4dd415d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 -- 172.21.15.105:0/2241885519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd415ed70 con 0x7f4dd409ba30 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd9437640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c7a0 tx=0x7f4dc400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:35.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4017070 con 0x7f4dd409ba30 2024-09-21T07:26:35.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dc4005070 con 0x7f4dd409ba30 2024-09-21T07:26:35.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd406e050 con 0x7f4dd409ba30 2024-09-21T07:26:35.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40054a0 con 0x7f4dd409ba30 2024-09-21T07:26:35.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.958+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd415f010 con 0x7f4dd409ba30 2024-09-21T07:26:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.959+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd40ccfe0 con 0x7f4dd409ba30 2024-09-21T07:26:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.960+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4dc401a070 con 0x7f4dd409ba30 2024-09-21T07:26:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.960+0000 7f4dd1ffb640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 0x7f4dc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.960+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4dc409a720 con 0x7f4dd409ba30 2024-09-21T07:26:35.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.961+0000 7f4dd8c36640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 0x7f4dc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.961+0000 7f4dd8c36640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 0x7f4dc007b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40d0620 tx=0x7f4dc8005be0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:35.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:35.965+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4dc40628c0 con 0x7f4dd409ba30 2024-09-21T07:26:36.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.179+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dd40b17b0 con 0x7f4dd409ba30 2024-09-21T07:26:36.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.179+0000 7f4dd1ffb640 1 -- 172.21.15.105:0/2241885519 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f4dc40621c0 con 0x7f4dd409ba30 2024-09-21T07:26:36.181 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:36.181 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:36.182 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:36.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 msgr2=0x7f4dc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 0x7f4dc007b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40d0620 tx=0x7f4dc8005be0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 msgr2=0x7f4dd415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c7a0 tx=0x7f4dc400cc60 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 shutdown_connections 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4dc0078b90 0x7f4dc007b0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4dd40cd520 0x7f4dd415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4dd40c59c0 0x7f4dd414b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 --2- 172.21.15.105:0/2241885519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd409ba30 0x7f4dd415e2c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.181+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 >> 172.21.15.105:0/2241885519 conn(0x7f4dd40cedb0 msgr2=0x7f4dd4072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.182+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 shutdown_connections 2024-09-21T07:26:36.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.182+0000 7f4ddb69b640 1 -- 172.21.15.105:0/2241885519 wait complete. 2024-09-21T07:26:36.194 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:36.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 -- 172.21.15.105:0/2137113196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780b34c0 msgr2=0x7f437815f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 --2- 172.21.15.105:0/2137113196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780b34c0 0x7f437815f1c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f436c009830 tx=0x7f436c02a5c0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 -- 172.21.15.105:0/2137113196 shutdown_connections 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 --2- 172.21.15.105:0/2137113196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f43780b34c0 0x7f437815f1c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 -- 172.21.15.105:0/2137113196 >> 172.21.15.105:0/2137113196 conn(0x7f43780b6610 msgr2=0x7f43780b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 -- 172.21.15.105:0/2137113196 shutdown_connections 2024-09-21T07:26:36.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.304+0000 7f437fd2b640 1 -- 172.21.15.105:0/2137113196 wait complete. 2024-09-21T07:26:36.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.306+0000 7f437fd2b640 1 Processor -- start 2024-09-21T07:26:36.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.306+0000 7f437fd2b640 1 -- start start 2024-09-21T07:26:36.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43780b34c0 0x7f437809e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437815cd90 0x7f4378153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4378166d30 con 0x7f437815f700 2024-09-21T07:26:36.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4378156040 con 0x7f437815cd90 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437fd2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f4378166eb0 con 0x7f43780b34c0 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.307+0000 7f437e2c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437d2c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437815cd90 0x7f4378153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437dac7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43780b34c0 0x7f437809e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59638/0 (socket says 172.21.15.105:59638) 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 -- 172.21.15.105:0/3643758701 learned_addr learned my addr 172.21.15.105:0/3643758701 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 -- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43780b34c0 msgr2=0x7f437809e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43780b34c0 0x7f437809e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 -- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437815cd90 msgr2=0x7f4378153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437815cd90 0x7f4378153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 -- 172.21.15.105:0/3643758701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43781f7ce0 con 0x7f437815f700 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437e2c8640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f437000c3c0 tx=0x7f437000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370017070 con 0x7f437815f700 2024-09-21T07:26:36.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43700053c0 con 0x7f437815f700 2024-09-21T07:26:36.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437809ebd0 con 0x7f437815f700 2024-09-21T07:26:36.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370003850 con 0x7f437815f700 2024-09-21T07:26:36.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.308+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43781f7200 con 0x7f437815f700 2024-09-21T07:26:36.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.309+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437815f1c0 con 0x7f437815f700 2024-09-21T07:26:36.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.310+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f437001a070 con 0x7f437815f700 2024-09-21T07:26:36.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.311+0000 7f43767fc640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 0x7f436407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.311+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f437009a310 con 0x7f437815f700 2024-09-21T07:26:36.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.311+0000 7f437dac7640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 0x7f436407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.311+0000 7f437dac7640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 0x7f436407b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43680012e0 tx=0x7f4368009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.315+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4370061f10 con 0x7f437815f700 2024-09-21T07:26:36.494 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.492+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f437809ee70 con 0x7f437815f700 2024-09-21T07:26:36.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.494+0000 7f43767fc640 1 -- 172.21.15.105:0/3643758701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f4370066940 con 0x7f437815f700 2024-09-21T07:26:36.496 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:36.496 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 msgr2=0x7f436407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 0x7f436407b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43680012e0 tx=0x7f4368009040 comp rx=0 tx=0).stop 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 msgr2=0x7f437806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f437000c3c0 tx=0x7f437000c880 comp rx=0 tx=0).stop 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 shutdown_connections 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4364078b90 0x7f436407b0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f437815cd90 0x7f4378153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f437815f700 0x7f437806cda0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 --2- 172.21.15.105:0/3643758701 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f43780b34c0 0x7f437809e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 >> 172.21.15.105:0/3643758701 conn(0x7f43780b6610 msgr2=0x7f4378070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 shutdown_connections 2024-09-21T07:26:36.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.496+0000 7f437fd2b640 1 -- 172.21.15.105:0/3643758701 wait complete. 2024-09-21T07:26:36.509 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:26:36.509 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:26:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 -- 172.21.15.105:0/1454944813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc074120 msgr2=0x7facbc074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 --2- 172.21.15.105:0/1454944813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc074120 0x7facbc074530 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7facbc077200 tx=0x7facb402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 -- 172.21.15.105:0/1454944813 shutdown_connections 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 --2- 172.21.15.105:0/1454944813 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc074120 0x7facbc074530 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 --2- 172.21.15.105:0/1454944813 >> v2:172.21.15.112:3300/0 conn(0x7facbc0701e0 0x7facbc15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 -- 172.21.15.105:0/1454944813 >> 172.21.15.105:0/1454944813 conn(0x7facbc0cdce0 msgr2=0x7facbc0ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 -- 172.21.15.105:0/1454944813 shutdown_connections 2024-09-21T07:26:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.619+0000 7facc3939640 1 -- 172.21.15.105:0/1454944813 wait complete. 2024-09-21T07:26:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.621+0000 7facc3939640 1 Processor -- start 2024-09-21T07:26:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.621+0000 7facc3939640 1 -- start start 2024-09-21T07:26:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 0x7facbc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 0x7facbc0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7facbc074bf0 con 0x7facbc06cda0 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facbc074a70 con 0x7facbc0701e0 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc3939640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7facbc074d70 con 0x7facbc074120 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.622+0000 7facc16d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59654/0 (socket says 172.21.15.105:59654) 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 -- 172.21.15.105:0/1340851982 learned_addr learned my addr 172.21.15.105:0/1340851982 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc0ed4640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 0x7facbc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc1ed6640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 0x7facbc0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 -- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 msgr2=0x7facbc0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 0x7facbc0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 -- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 msgr2=0x7facbc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 0x7facbc15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 -- 172.21.15.105:0/1340851982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc0c7070 con 0x7facbc06cda0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc1ed6640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 0x7facbc0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc0ed4640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 0x7facbc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc16d5640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7facb800e790 tx=0x7facb800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb800eea0 con 0x7facbc06cda0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facb800cd50 con 0x7facbc06cda0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facbc0c4f30 con 0x7facbc06cda0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8010690 con 0x7facbc06cda0 2024-09-21T07:26:36.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.623+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facbc0c54f0 con 0x7facbc06cda0 2024-09-21T07:26:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.624+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facbc074890 con 0x7facbc06cda0 2024-09-21T07:26:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.625+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7facb8010830 con 0x7facbc06cda0 2024-09-21T07:26:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.626+0000 7facb1ffb640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 0x7faca007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.626+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7facb809ba20 con 0x7facbc06cda0 2024-09-21T07:26:36.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.626+0000 7facc0ed4640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 0x7faca007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.627+0000 7facc0ed4640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 0x7faca007b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facbc0b3d90 tx=0x7facac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.631+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7facbc074890 con 0x7facbc06cda0 2024-09-21T07:26:36.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.806+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7facbc0c4d60 con 0x7faca0078bc0 2024-09-21T07:26:36.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.813+0000 7facb1ffb640 1 -- 172.21.15.105:0/1340851982 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7facbc0c4d60 con 0x7faca0078bc0 2024-09-21T07:26:36.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:36.820 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:26:36.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 msgr2=0x7faca007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 0x7faca007b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7facbc0b3d90 tx=0x7facac009040 comp rx=0 tx=0).stop 2024-09-21T07:26:36.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 msgr2=0x7facbc1f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7facb800e790 tx=0x7facb800ec50 comp rx=0 tx=0).stop 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 shutdown_connections 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faca0078bc0 0x7faca007b0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facbc0701e0 0x7facbc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7facbc074120 0x7facbc0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 --2- 172.21.15.105:0/1340851982 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7facbc06cda0 0x7facbc1f6680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.816+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 >> 172.21.15.105:0/1340851982 conn(0x7facbc0cdce0 msgr2=0x7facbc0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.817+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 shutdown_connections 2024-09-21T07:26:36.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.817+0000 7facc3939640 1 -- 172.21.15.105:0/1340851982 wait complete. 2024-09-21T07:26:36.829 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T07:26:36.770989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3514368,"data_stored":2101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003728"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755382+0000","last_change":"2024-09-21T07:26:14.746008+0000","last_active":"2024-09-21T07:26:22.755382+0000","last_peered":"2024-09-21T07:26:22.755382+0000","last_clean":"2024-09-21T07:26:22.755382+0000","last_became_active":"2024-09-21T07:26:14.745685+0000","last_became_peered":"2024-09-21T07:26:14.745685+0000","last_unstale":"2024-09-21T07:26:22.755382+0000","last_undegraded":"2024-09-21T07:26:22.755382+0000","last_fullsized":"2024-09-21T07:26:22.755382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:02.797953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217709+0000","last_change":"2024-09-21T07:26:15.767737+0000","last_active":"2024-09-21T07:26:23.217709+0000","last_peered":"2024-09-21T07:26:23.217709+0000","last_clean":"2024-09-21T07:26:23.217709+0000","last_became_active":"2024-09-21T07:26:15.767112+0000","last_became_peered":"2024-09-21T07:26:15.767112+0000","last_unstale":"2024-09-21T07:26:23.217709+0000","last_undegraded":"2024-09-21T07:26:23.217709+0000","last_fullsized":"2024-09-21T07:26:23.217709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.781454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217413+0000","last_change":"2024-09-21T07:26:20.205421+0000","last_active":"2024-09-21T07:26:23.217413+0000","last_peered":"2024-09-21T07:26:23.217413+0000","last_clean":"2024-09-21T07:26:23.217413+0000","last_became_active":"2024-09-21T07:26:20.204943+0000","last_became_peered":"2024-09-21T07:26:20.204943+0000","last_unstale":"2024-09-21T07:26:23.217413+0000","last_undegraded":"2024-09-21T07:26:23.217413+0000","last_fullsized":"2024-09-21T07:26:23.217413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:55.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:14.738042+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:14.737714+0000","last_became_peered":"2024-09-21T07:26:14.737714+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:43.999074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756065+0000","last_change":"2024-09-21T07:26:15.760507+0000","last_active":"2024-09-21T07:26:22.756065+0000","last_peered":"2024-09-21T07:26:22.756065+0000","last_clean":"2024-09-21T07:26:22.756065+0000","last_became_active":"2024-09-21T07:26:15.759859+0000","last_became_peered":"2024-09-21T07:26:15.759859+0000","last_unstale":"2024-09-21T07:26:22.756065+0000","last_undegraded":"2024-09-21T07:26:22.756065+0000","last_fullsized":"2024-09-21T07:26:22.756065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217096+0000","last_change":"2024-09-21T07:26:19.783194+0000","last_active":"2024-09-21T07:26:23.217096+0000","last_peered":"2024-09-21T07:26:23.217096+0000","last_clean":"2024-09-21T07:26:23.217096+0000","last_became_active":"2024-09-21T07:26:19.782625+0000","last_became_peered":"2024-09-21T07:26:19.782625+0000","last_unstale":"2024-09-21T07:26:23.217096+0000","last_undegraded":"2024-09-21T07:26:23.217096+0000","last_fullsized":"2024-09-21T07:26:23.217096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791768+0000","last_change":"2024-09-21T07:26:14.748310+0000","last_active":"2024-09-21T07:26:21.791768+0000","last_peered":"2024-09-21T07:26:21.791768+0000","last_clean":"2024-09-21T07:26:21.791768+0000","last_became_active":"2024-09-21T07:26:14.747943+0000","last_became_peered":"2024-09-21T07:26:14.747943+0000","last_unstale":"2024-09-21T07:26:21.791768+0000","last_undegraded":"2024-09-21T07:26:21.791768+0000","last_fullsized":"2024-09-21T07:26:21.791768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:12.833502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901017+0000","last_change":"2024-09-21T07:26:15.772109+0000","last_active":"2024-09-21T07:26:22.901017+0000","last_peered":"2024-09-21T07:26:22.901017+0000","last_clean":"2024-09-21T07:26:22.901017+0000","last_became_active":"2024-09-21T07:26:15.771762+0000","last_became_peered":"2024-09-21T07:26:15.771762+0000","last_unstale":"2024-09-21T07:26:22.901017+0000","last_undegraded":"2024-09-21T07:26:22.901017+0000","last_fullsized":"2024-09-21T07:26:22.901017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:14.024694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217587+0000","last_change":"2024-09-21T07:26:19.782636+0000","last_active":"2024-09-21T07:26:23.217587+0000","last_peered":"2024-09-21T07:26:23.217587+0000","last_clean":"2024-09-21T07:26:23.217587+0000","last_became_active":"2024-09-21T07:26:19.782148+0000","last_became_peered":"2024-09-21T07:26:19.782148+0000","last_unstale":"2024-09-21T07:26:23.217587+0000","last_undegraded":"2024-09-21T07:26:23.217587+0000","last_fullsized":"2024-09-21T07:26:23.217587+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:34.102740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791003+0000","last_change":"2024-09-21T07:26:14.756864+0000","last_active":"2024-09-21T07:26:21.791003+0000","last_peered":"2024-09-21T07:26:21.791003+0000","last_clean":"2024-09-21T07:26:21.791003+0000","last_became_active":"2024-09-21T07:26:14.756519+0000","last_became_peered":"2024-09-21T07:26:14.756519+0000","last_unstale":"2024-09-21T07:26:21.791003+0000","last_undegraded":"2024-09-21T07:26:21.791003+0000","last_fullsized":"2024-09-21T07:26:21.791003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.306611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901081+0000","last_change":"2024-09-21T07:26:15.769890+0000","last_active":"2024-09-21T07:26:22.901081+0000","last_peered":"2024-09-21T07:26:22.901081+0000","last_clean":"2024-09-21T07:26:22.901081+0000","last_became_active":"2024-09-21T07:26:15.769124+0000","last_became_peered":"2024-09-21T07:26:15.769124+0000","last_unstale":"2024-09-21T07:26:22.901081+0000","last_undegraded":"2024-09-21T07:26:22.901081+0000","last_fullsized":"2024-09-21T07:26:22.901081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:55.122043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217329+0000","last_change":"2024-09-21T07:26:19.788443+0000","last_active":"2024-09-21T07:26:23.217329+0000","last_peered":"2024-09-21T07:26:23.217329+0000","last_clean":"2024-09-21T07:26:23.217329+0000","last_became_active":"2024-09-21T07:26:19.787922+0000","last_became_peered":"2024-09-21T07:26:19.787922+0000","last_unstale":"2024-09-21T07:26:23.217329+0000","last_undegraded":"2024-09-21T07:26:23.217329+0000","last_fullsized":"2024-09-21T07:26:23.217329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790557+0000","last_change":"2024-09-21T07:26:15.774537+0000","last_active":"2024-09-21T07:26:21.790557+0000","last_peered":"2024-09-21T07:26:21.790557+0000","last_clean":"2024-09-21T07:26:21.790557+0000","last_became_active":"2024-09-21T07:26:15.774215+0000","last_became_peered":"2024-09-21T07:26:15.774215+0000","last_unstale":"2024-09-21T07:26:21.790557+0000","last_undegraded":"2024-09-21T07:26:21.790557+0000","last_fullsized":"2024-09-21T07:26:21.790557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791990+0000","last_change":"2024-09-21T07:26:14.752255+0000","last_active":"2024-09-21T07:26:21.791990+0000","last_peered":"2024-09-21T07:26:21.791990+0000","last_clean":"2024-09-21T07:26:21.791990+0000","last_became_active":"2024-09-21T07:26:14.751809+0000","last_became_peered":"2024-09-21T07:26:14.751809+0000","last_unstale":"2024-09-21T07:26:21.791990+0000","last_undegraded":"2024-09-21T07:26:21.791990+0000","last_fullsized":"2024-09-21T07:26:21.791990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:30.055064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218154+0000","last_change":"2024-09-21T07:26:19.784193+0000","last_active":"2024-09-21T07:26:23.218154+0000","last_peered":"2024-09-21T07:26:23.218154+0000","last_clean":"2024-09-21T07:26:23.218154+0000","last_became_active":"2024-09-21T07:26:19.783666+0000","last_became_peered":"2024-09-21T07:26:19.783666+0000","last_unstale":"2024-09-21T07:26:23.218154+0000","last_undegraded":"2024-09-21T07:26:23.218154+0000","last_fullsized":"2024-09-21T07:26:23.218154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218051+0000","last_change":"2024-09-21T07:26:15.761293+0000","last_active":"2024-09-21T07:26:23.218051+0000","last_peered":"2024-09-21T07:26:23.218051+0000","last_clean":"2024-09-21T07:26:23.218051+0000","last_became_active":"2024-09-21T07:26:15.760959+0000","last_became_peered":"2024-09-21T07:26:15.760959+0000","last_unstale":"2024-09-21T07:26:23.218051+0000","last_undegraded":"2024-09-21T07:26:23.218051+0000","last_fullsized":"2024-09-21T07:26:23.218051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:12.766792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218529+0000","last_change":"2024-09-21T07:26:14.737676+0000","last_active":"2024-09-21T07:26:23.218529+0000","last_peered":"2024-09-21T07:26:23.218529+0000","last_clean":"2024-09-21T07:26:23.218529+0000","last_became_active":"2024-09-21T07:26:14.737367+0000","last_became_peered":"2024-09-21T07:26:14.737367+0000","last_unstale":"2024-09-21T07:26:23.218529+0000","last_undegraded":"2024-09-21T07:26:23.218529+0000","last_fullsized":"2024-09-21T07:26:23.218529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217727+0000","last_change":"2024-09-21T07:26:19.782689+0000","last_active":"2024-09-21T07:26:23.217727+0000","last_peered":"2024-09-21T07:26:23.217727+0000","last_clean":"2024-09-21T07:26:23.217727+0000","last_became_active":"2024-09-21T07:26:19.782215+0000","last_became_peered":"2024-09-21T07:26:19.782215+0000","last_unstale":"2024-09-21T07:26:23.217727+0000","last_undegraded":"2024-09-21T07:26:23.217727+0000","last_fullsized":"2024-09-21T07:26:23.217727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218798+0000","last_change":"2024-09-21T07:26:15.758262+0000","last_active":"2024-09-21T07:26:23.218798+0000","last_peered":"2024-09-21T07:26:23.218798+0000","last_clean":"2024-09-21T07:26:23.218798+0000","last_became_active":"2024-09-21T07:26:15.757944+0000","last_became_peered":"2024-09-21T07:26:15.757944+0000","last_unstale":"2024-09-21T07:26:23.218798+0000","last_undegraded":"2024-09-21T07:26:23.218798+0000","last_fullsized":"2024-09-21T07:26:23.218798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:10.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216341+0000","last_change":"2024-09-21T07:26:14.745239+0000","last_active":"2024-09-21T07:26:23.216341+0000","last_peered":"2024-09-21T07:26:23.216341+0000","last_clean":"2024-09-21T07:26:23.216341+0000","last_became_active":"2024-09-21T07:26:14.744901+0000","last_became_peered":"2024-09-21T07:26:14.744901+0000","last_unstale":"2024-09-21T07:26:23.216341+0000","last_undegraded":"2024-09-21T07:26:23.216341+0000","last_fullsized":"2024-09-21T07:26:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:07.561158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218435+0000","last_change":"2024-09-21T07:26:20.205294+0000","last_active":"2024-09-21T07:26:23.218435+0000","last_peered":"2024-09-21T07:26:23.218435+0000","last_clean":"2024-09-21T07:26:23.218435+0000","last_became_active":"2024-09-21T07:26:20.204855+0000","last_became_peered":"2024-09-21T07:26:20.204855+0000","last_unstale":"2024-09-21T07:26:23.218435+0000","last_undegraded":"2024-09-21T07:26:23.218435+0000","last_fullsized":"2024-09-21T07:26:23.218435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217428+0000","last_change":"2024-09-21T07:26:15.762544+0000","last_active":"2024-09-21T07:26:23.217428+0000","last_peered":"2024-09-21T07:26:23.217428+0000","last_clean":"2024-09-21T07:26:23.217428+0000","last_became_active":"2024-09-21T07:26:15.762193+0000","last_became_peered":"2024-09-21T07:26:15.762193+0000","last_unstale":"2024-09-21T07:26:23.217428+0000","last_undegraded":"2024-09-21T07:26:23.217428+0000","last_fullsized":"2024-09-21T07:26:23.217428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:42.681607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790143+0000","last_change":"2024-09-21T07:26:14.748084+0000","last_active":"2024-09-21T07:26:21.790143+0000","last_peered":"2024-09-21T07:26:21.790143+0000","last_clean":"2024-09-21T07:26:21.790143+0000","last_became_active":"2024-09-21T07:26:14.747288+0000","last_became_peered":"2024-09-21T07:26:14.747288+0000","last_unstale":"2024-09-21T07:26:21.790143+0000","last_undegraded":"2024-09-21T07:26:21.790143+0000","last_fullsized":"2024-09-21T07:26:21.790143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:39.950583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:19.788640+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:19.788072+0000","last_became_peered":"2024-09-21T07:26:19.788072+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756286+0000","last_change":"2024-09-21T07:26:15.757686+0000","last_active":"2024-09-21T07:26:22.756286+0000","last_peered":"2024-09-21T07:26:22.756286+0000","last_clean":"2024-09-21T07:26:22.756286+0000","last_became_active":"2024-09-21T07:26:15.757361+0000","last_became_peered":"2024-09-21T07:26:15.757361+0000","last_unstale":"2024-09-21T07:26:22.756286+0000","last_undegraded":"2024-09-21T07:26:22.756286+0000","last_fullsized":"2024-09-21T07:26:22.756286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:49.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791712+0000","last_change":"2024-09-21T07:26:14.746130+0000","last_active":"2024-09-21T07:26:21.791712+0000","last_peered":"2024-09-21T07:26:21.791712+0000","last_clean":"2024-09-21T07:26:21.791712+0000","last_became_active":"2024-09-21T07:26:14.745763+0000","last_became_peered":"2024-09-21T07:26:14.745763+0000","last_unstale":"2024-09-21T07:26:21.791712+0000","last_undegraded":"2024-09-21T07:26:21.791712+0000","last_fullsized":"2024-09-21T07:26:21.791712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217052+0000","last_change":"2024-09-21T07:26:20.203703+0000","last_active":"2024-09-21T07:26:23.217052+0000","last_peered":"2024-09-21T07:26:23.217052+0000","last_clean":"2024-09-21T07:26:23.217052+0000","last_became_active":"2024-09-21T07:26:20.202561+0000","last_became_peered":"2024-09-21T07:26:20.202561+0000","last_unstale":"2024-09-21T07:26:23.217052+0000","last_undegraded":"2024-09-21T07:26:23.217052+0000","last_fullsized":"2024-09-21T07:26:23.217052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:27.041439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218313+0000","last_change":"2024-09-21T07:26:15.768378+0000","last_active":"2024-09-21T07:26:23.218313+0000","last_peered":"2024-09-21T07:26:23.218313+0000","last_clean":"2024-09-21T07:26:23.218313+0000","last_became_active":"2024-09-21T07:26:15.768068+0000","last_became_peered":"2024-09-21T07:26:15.768068+0000","last_unstale":"2024-09-21T07:26:23.218313+0000","last_undegraded":"2024-09-21T07:26:23.218313+0000","last_fullsized":"2024-09-21T07:26:23.218313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:12.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216461+0000","last_change":"2024-09-21T07:26:14.741727+0000","last_active":"2024-09-21T07:26:23.216461+0000","last_peered":"2024-09-21T07:26:23.216461+0000","last_clean":"2024-09-21T07:26:23.216461+0000","last_became_active":"2024-09-21T07:26:14.741372+0000","last_became_peered":"2024-09-21T07:26:14.741372+0000","last_unstale":"2024-09-21T07:26:23.216461+0000","last_undegraded":"2024-09-21T07:26:23.216461+0000","last_fullsized":"2024-09-21T07:26:23.216461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:48.123776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218506+0000","last_change":"2024-09-21T07:26:20.203003+0000","last_active":"2024-09-21T07:26:23.218506+0000","last_peered":"2024-09-21T07:26:23.218506+0000","last_clean":"2024-09-21T07:26:23.218506+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:23.218506+0000","last_undegraded":"2024-09-21T07:26:23.218506+0000","last_fullsized":"2024-09-21T07:26:23.218506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:12.749885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217922+0000","last_change":"2024-09-21T07:26:15.750763+0000","last_active":"2024-09-21T07:26:23.217922+0000","last_peered":"2024-09-21T07:26:23.217922+0000","last_clean":"2024-09-21T07:26:23.217922+0000","last_became_active":"2024-09-21T07:26:15.750444+0000","last_became_peered":"2024-09-21T07:26:15.750444+0000","last_unstale":"2024-09-21T07:26:23.217922+0000","last_undegraded":"2024-09-21T07:26:23.217922+0000","last_fullsized":"2024-09-21T07:26:23.217922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792041+0000","last_change":"2024-09-21T07:26:14.745457+0000","last_active":"2024-09-21T07:26:21.792041+0000","last_peered":"2024-09-21T07:26:21.792041+0000","last_clean":"2024-09-21T07:26:21.792041+0000","last_became_active":"2024-09-21T07:26:14.745075+0000","last_became_peered":"2024-09-21T07:26:14.745075+0000","last_unstale":"2024-09-21T07:26:21.792041+0000","last_undegraded":"2024-09-21T07:26:21.792041+0000","last_fullsized":"2024-09-21T07:26:21.792041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:49.174372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756061+0000","last_change":"2024-09-21T07:26:20.202406+0000","last_active":"2024-09-21T07:26:22.756061+0000","last_peered":"2024-09-21T07:26:22.756061+0000","last_clean":"2024-09-21T07:26:22.756061+0000","last_became_active":"2024-09-21T07:26:20.201970+0000","last_became_peered":"2024-09-21T07:26:20.201970+0000","last_unstale":"2024-09-21T07:26:22.756061+0000","last_undegraded":"2024-09-21T07:26:22.756061+0000","last_fullsized":"2024-09-21T07:26:22.756061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901194+0000","last_change":"2024-09-21T07:26:15.769492+0000","last_active":"2024-09-21T07:26:22.901194+0000","last_peered":"2024-09-21T07:26:22.901194+0000","last_clean":"2024-09-21T07:26:22.901194+0000","last_became_active":"2024-09-21T07:26:15.768661+0000","last_became_peered":"2024-09-21T07:26:15.768661+0000","last_unstale":"2024-09-21T07:26:22.901194+0000","last_undegraded":"2024-09-21T07:26:22.901194+0000","last_fullsized":"2024-09-21T07:26:22.901194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:20.108036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218024+0000","last_change":"2024-09-21T07:26:14.737203+0000","last_active":"2024-09-21T07:26:23.218024+0000","last_peered":"2024-09-21T07:26:23.218024+0000","last_clean":"2024-09-21T07:26:23.218024+0000","last_became_active":"2024-09-21T07:26:14.736387+0000","last_became_peered":"2024-09-21T07:26:14.736387+0000","last_unstale":"2024-09-21T07:26:23.218024+0000","last_undegraded":"2024-09-21T07:26:23.218024+0000","last_fullsized":"2024-09-21T07:26:23.218024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217332+0000","last_change":"2024-09-21T07:26:19.793567+0000","last_active":"2024-09-21T07:26:23.217332+0000","last_peered":"2024-09-21T07:26:23.217332+0000","last_clean":"2024-09-21T07:26:23.217332+0000","last_became_active":"2024-09-21T07:26:19.793063+0000","last_became_peered":"2024-09-21T07:26:19.793063+0000","last_unstale":"2024-09-21T07:26:23.217332+0000","last_undegraded":"2024-09-21T07:26:23.217332+0000","last_fullsized":"2024-09-21T07:26:23.217332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:15.764293+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:15.763942+0000","last_became_peered":"2024-09-21T07:26:15.763942+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:40.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900887+0000","last_change":"2024-09-21T07:26:14.753757+0000","last_active":"2024-09-21T07:26:22.900887+0000","last_peered":"2024-09-21T07:26:22.900887+0000","last_clean":"2024-09-21T07:26:22.900887+0000","last_became_active":"2024-09-21T07:26:14.753415+0000","last_became_peered":"2024-09-21T07:26:14.753415+0000","last_unstale":"2024-09-21T07:26:22.900887+0000","last_undegraded":"2024-09-21T07:26:22.900887+0000","last_fullsized":"2024-09-21T07:26:22.900887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:48.397593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217762+0000","last_change":"2024-09-21T07:26:19.786373+0000","last_active":"2024-09-21T07:26:23.217762+0000","last_peered":"2024-09-21T07:26:23.217762+0000","last_clean":"2024-09-21T07:26:23.217762+0000","last_became_active":"2024-09-21T07:26:19.785794+0000","last_became_peered":"2024-09-21T07:26:19.785794+0000","last_unstale":"2024-09-21T07:26:23.217762+0000","last_undegraded":"2024-09-21T07:26:23.217762+0000","last_fullsized":"2024-09-21T07:26:23.217762+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216872+0000","last_change":"2024-09-21T07:26:15.752034+0000","last_active":"2024-09-21T07:26:23.216872+0000","last_peered":"2024-09-21T07:26:23.216872+0000","last_clean":"2024-09-21T07:26:23.216872+0000","last_became_active":"2024-09-21T07:26:15.751733+0000","last_became_peered":"2024-09-21T07:26:15.751733+0000","last_unstale":"2024-09-21T07:26:23.216872+0000","last_undegraded":"2024-09-21T07:26:23.216872+0000","last_fullsized":"2024-09-21T07:26:23.216872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789086+0000","last_change":"2024-09-21T07:26:14.741318+0000","last_active":"2024-09-21T07:26:21.789086+0000","last_peered":"2024-09-21T07:26:21.789086+0000","last_clean":"2024-09-21T07:26:21.789086+0000","last_became_active":"2024-09-21T07:26:14.740828+0000","last_became_peered":"2024-09-21T07:26:14.740828+0000","last_unstale":"2024-09-21T07:26:21.789086+0000","last_undegraded":"2024-09-21T07:26:21.789086+0000","last_fullsized":"2024-09-21T07:26:21.789086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.614543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218456+0000","last_change":"2024-09-21T07:26:19.793288+0000","last_active":"2024-09-21T07:26:23.218456+0000","last_peered":"2024-09-21T07:26:23.218456+0000","last_clean":"2024-09-21T07:26:23.218456+0000","last_became_active":"2024-09-21T07:26:19.792781+0000","last_became_peered":"2024-09-21T07:26:19.792781+0000","last_unstale":"2024-09-21T07:26:23.218456+0000","last_undegraded":"2024-09-21T07:26:23.218456+0000","last_fullsized":"2024-09-21T07:26:23.218456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755586+0000","last_change":"2024-09-21T07:26:15.763289+0000","last_active":"2024-09-21T07:26:22.755586+0000","last_peered":"2024-09-21T07:26:22.755586+0000","last_clean":"2024-09-21T07:26:22.755586+0000","last_became_active":"2024-09-21T07:26:15.762914+0000","last_became_peered":"2024-09-21T07:26:15.762914+0000","last_unstale":"2024-09-21T07:26:22.755586+0000","last_undegraded":"2024-09-21T07:26:22.755586+0000","last_fullsized":"2024-09-21T07:26:22.755586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:01.890170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790269+0000","last_change":"2024-09-21T07:26:14.746676+0000","last_active":"2024-09-21T07:26:21.790269+0000","last_peered":"2024-09-21T07:26:21.790269+0000","last_clean":"2024-09-21T07:26:21.790269+0000","last_became_active":"2024-09-21T07:26:14.745094+0000","last_became_peered":"2024-09-21T07:26:14.745094+0000","last_unstale":"2024-09-21T07:26:21.790269+0000","last_undegraded":"2024-09-21T07:26:21.790269+0000","last_fullsized":"2024-09-21T07:26:21.790269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:48.031389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756099+0000","last_change":"2024-09-21T07:26:20.202272+0000","last_active":"2024-09-21T07:26:22.756099+0000","last_peered":"2024-09-21T07:26:22.756099+0000","last_clean":"2024-09-21T07:26:22.756099+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:22.756099+0000","last_undegraded":"2024-09-21T07:26:22.756099+0000","last_fullsized":"2024-09-21T07:26:22.756099+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217063+0000","last_change":"2024-09-21T07:26:15.778494+0000","last_active":"2024-09-21T07:26:23.217063+0000","last_peered":"2024-09-21T07:26:23.217063+0000","last_clean":"2024-09-21T07:26:23.217063+0000","last_became_active":"2024-09-21T07:26:15.778117+0000","last_became_peered":"2024-09-21T07:26:15.778117+0000","last_unstale":"2024-09-21T07:26:23.217063+0000","last_undegraded":"2024-09-21T07:26:23.217063+0000","last_fullsized":"2024-09-21T07:26:23.217063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:52.614175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900190+0000","last_change":"2024-09-21T07:26:14.755827+0000","last_active":"2024-09-21T07:26:22.900190+0000","last_peered":"2024-09-21T07:26:22.900190+0000","last_clean":"2024-09-21T07:26:22.900190+0000","last_became_active":"2024-09-21T07:26:14.755319+0000","last_became_peered":"2024-09-21T07:26:14.755319+0000","last_unstale":"2024-09-21T07:26:22.900190+0000","last_undegraded":"2024-09-21T07:26:22.900190+0000","last_fullsized":"2024-09-21T07:26:22.900190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:31.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790381+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:26:21.790381+0000","last_peered":"2024-09-21T07:26:21.790381+0000","last_clean":"2024-09-21T07:26:21.790381+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:26:21.790381+0000","last_undegraded":"2024-09-21T07:26:21.790381+0000","last_fullsized":"2024-09-21T07:26:21.790381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790882+0000","last_change":"2024-09-21T07:26:20.205543+0000","last_active":"2024-09-21T07:26:21.790882+0000","last_peered":"2024-09-21T07:26:21.790882+0000","last_clean":"2024-09-21T07:26:21.790882+0000","last_became_active":"2024-09-21T07:26:20.204540+0000","last_became_peered":"2024-09-21T07:26:20.204540+0000","last_unstale":"2024-09-21T07:26:21.790882+0000","last_undegraded":"2024-09-21T07:26:21.790882+0000","last_fullsized":"2024-09-21T07:26:21.790882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791526+0000","last_change":"2024-09-21T07:26:15.776215+0000","last_active":"2024-09-21T07:26:21.791526+0000","last_peered":"2024-09-21T07:26:21.791526+0000","last_clean":"2024-09-21T07:26:21.791526+0000","last_became_active":"2024-09-21T07:26:15.775884+0000","last_became_peered":"2024-09-21T07:26:15.775884+0000","last_unstale":"2024-09-21T07:26:21.791526+0000","last_undegraded":"2024-09-21T07:26:21.791526+0000","last_fullsized":"2024-09-21T07:26:21.791526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:01.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789487+0000","last_change":"2024-09-21T07:26:14.756009+0000","last_active":"2024-09-21T07:26:21.789487+0000","last_peered":"2024-09-21T07:26:21.789487+0000","last_clean":"2024-09-21T07:26:21.789487+0000","last_became_active":"2024-09-21T07:26:14.755653+0000","last_became_peered":"2024-09-21T07:26:14.755653+0000","last_unstale":"2024-09-21T07:26:21.789487+0000","last_undegraded":"2024-09-21T07:26:21.789487+0000","last_fullsized":"2024-09-21T07:26:21.789487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:22.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790353+0000","last_change":"2024-09-21T07:26:20.202159+0000","last_active":"2024-09-21T07:26:21.790353+0000","last_peered":"2024-09-21T07:26:21.790353+0000","last_clean":"2024-09-21T07:26:21.790353+0000","last_became_active":"2024-09-21T07:26:20.201680+0000","last_became_peered":"2024-09-21T07:26:20.201680+0000","last_unstale":"2024-09-21T07:26:21.790353+0000","last_undegraded":"2024-09-21T07:26:21.790353+0000","last_fullsized":"2024-09-21T07:26:21.790353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:56.792767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901045+0000","last_change":"2024-09-21T07:26:15.771710+0000","last_active":"2024-09-21T07:26:22.901045+0000","last_peered":"2024-09-21T07:26:22.901045+0000","last_clean":"2024-09-21T07:26:22.901045+0000","last_became_active":"2024-09-21T07:26:15.770649+0000","last_became_peered":"2024-09-21T07:26:15.770649+0000","last_unstale":"2024-09-21T07:26:22.901045+0000","last_undegraded":"2024-09-21T07:26:22.901045+0000","last_fullsized":"2024-09-21T07:26:22.901045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.904168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218377+0000","last_change":"2024-09-21T07:26:14.740185+0000","last_active":"2024-09-21T07:26:23.218377+0000","last_peered":"2024-09-21T07:26:23.218377+0000","last_clean":"2024-09-21T07:26:23.218377+0000","last_became_active":"2024-09-21T07:26:14.738935+0000","last_became_peered":"2024-09-21T07:26:14.738935+0000","last_unstale":"2024-09-21T07:26:23.218377+0000","last_undegraded":"2024-09-21T07:26:23.218377+0000","last_fullsized":"2024-09-21T07:26:23.218377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756214+0000","last_change":"2024-09-21T07:26:19.789320+0000","last_active":"2024-09-21T07:26:22.756214+0000","last_peered":"2024-09-21T07:26:22.756214+0000","last_clean":"2024-09-21T07:26:22.756214+0000","last_became_active":"2024-09-21T07:26:19.788692+0000","last_became_peered":"2024-09-21T07:26:19.788692+0000","last_unstale":"2024-09-21T07:26:22.756214+0000","last_undegraded":"2024-09-21T07:26:22.756214+0000","last_fullsized":"2024-09-21T07:26:22.756214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792007+0000","last_change":"2024-09-21T07:26:15.775282+0000","last_active":"2024-09-21T07:26:21.792007+0000","last_peered":"2024-09-21T07:26:21.792007+0000","last_clean":"2024-09-21T07:26:21.792007+0000","last_became_active":"2024-09-21T07:26:15.775046+0000","last_became_peered":"2024-09-21T07:26:15.775046+0000","last_unstale":"2024-09-21T07:26:21.792007+0000","last_undegraded":"2024-09-21T07:26:21.792007+0000","last_fullsized":"2024-09-21T07:26:21.792007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:51.588092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789401+0000","last_change":"2024-09-21T07:26:14.740365+0000","last_active":"2024-09-21T07:26:21.789401+0000","last_peered":"2024-09-21T07:26:21.789401+0000","last_clean":"2024-09-21T07:26:21.789401+0000","last_became_active":"2024-09-21T07:26:14.739987+0000","last_became_peered":"2024-09-21T07:26:14.739987+0000","last_unstale":"2024-09-21T07:26:21.789401+0000","last_undegraded":"2024-09-21T07:26:21.789401+0000","last_fullsized":"2024-09-21T07:26:21.789401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900421+0000","last_change":"2024-09-21T07:26:20.205672+0000","last_active":"2024-09-21T07:26:22.900421+0000","last_peered":"2024-09-21T07:26:22.900421+0000","last_clean":"2024-09-21T07:26:22.900421+0000","last_became_active":"2024-09-21T07:26:20.205251+0000","last_became_peered":"2024-09-21T07:26:20.205251+0000","last_unstale":"2024-09-21T07:26:22.900421+0000","last_undegraded":"2024-09-21T07:26:22.900421+0000","last_fullsized":"2024-09-21T07:26:22.900421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:18.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790765+0000","last_change":"2024-09-21T07:26:15.762918+0000","last_active":"2024-09-21T07:26:21.790765+0000","last_peered":"2024-09-21T07:26:21.790765+0000","last_clean":"2024-09-21T07:26:21.790765+0000","last_became_active":"2024-09-21T07:26:15.762580+0000","last_became_peered":"2024-09-21T07:26:15.762580+0000","last_unstale":"2024-09-21T07:26:21.790765+0000","last_undegraded":"2024-09-21T07:26:21.790765+0000","last_fullsized":"2024-09-21T07:26:21.790765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.691687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218021+0000","last_change":"2024-09-21T07:26:14.739242+0000","last_active":"2024-09-21T07:26:23.218021+0000","last_peered":"2024-09-21T07:26:23.218021+0000","last_clean":"2024-09-21T07:26:23.218021+0000","last_became_active":"2024-09-21T07:26:14.737910+0000","last_became_peered":"2024-09-21T07:26:14.737910+0000","last_unstale":"2024-09-21T07:26:23.218021+0000","last_undegraded":"2024-09-21T07:26:23.218021+0000","last_fullsized":"2024-09-21T07:26:23.218021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:30.239449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217037+0000","last_change":"2024-09-21T07:26:19.782735+0000","last_active":"2024-09-21T07:26:23.217037+0000","last_peered":"2024-09-21T07:26:23.217037+0000","last_clean":"2024-09-21T07:26:23.217037+0000","last_became_active":"2024-09-21T07:26:19.782062+0000","last_became_peered":"2024-09-21T07:26:19.782062+0000","last_unstale":"2024-09-21T07:26:23.217037+0000","last_undegraded":"2024-09-21T07:26:23.217037+0000","last_fullsized":"2024-09-21T07:26:23.217037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790993+0000","last_change":"2024-09-21T07:26:15.772188+0000","last_active":"2024-09-21T07:26:21.790993+0000","last_peered":"2024-09-21T07:26:21.790993+0000","last_clean":"2024-09-21T07:26:21.790993+0000","last_became_active":"2024-09-21T07:26:15.771854+0000","last_became_peered":"2024-09-21T07:26:15.771854+0000","last_unstale":"2024-09-21T07:26:21.790993+0000","last_undegraded":"2024-09-21T07:26:21.790993+0000","last_fullsized":"2024-09-21T07:26:21.790993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:52.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.788951+0000","last_change":"2024-09-21T07:26:14.739783+0000","last_active":"2024-09-21T07:26:21.788951+0000","last_peered":"2024-09-21T07:26:21.788951+0000","last_clean":"2024-09-21T07:26:21.788951+0000","last_became_active":"2024-09-21T07:26:14.739433+0000","last_became_peered":"2024-09-21T07:26:14.739433+0000","last_unstale":"2024-09-21T07:26:21.788951+0000","last_undegraded":"2024-09-21T07:26:21.788951+0000","last_fullsized":"2024-09-21T07:26:21.788951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:08.827173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756045+0000","last_change":"2024-09-21T07:26:19.789353+0000","last_active":"2024-09-21T07:26:22.756045+0000","last_peered":"2024-09-21T07:26:22.756045+0000","last_clean":"2024-09-21T07:26:22.756045+0000","last_became_active":"2024-09-21T07:26:19.788890+0000","last_became_peered":"2024-09-21T07:26:19.788890+0000","last_unstale":"2024-09-21T07:26:22.756045+0000","last_undegraded":"2024-09-21T07:26:22.756045+0000","last_fullsized":"2024-09-21T07:26:22.756045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790254+0000","last_change":"2024-09-21T07:26:15.761846+0000","last_active":"2024-09-21T07:26:21.790254+0000","last_peered":"2024-09-21T07:26:21.790254+0000","last_clean":"2024-09-21T07:26:21.790254+0000","last_became_active":"2024-09-21T07:26:15.761125+0000","last_became_peered":"2024-09-21T07:26:15.761125+0000","last_unstale":"2024-09-21T07:26:21.790254+0000","last_undegraded":"2024-09-21T07:26:21.790254+0000","last_fullsized":"2024-09-21T07:26:21.790254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791303+0000","last_change":"2024-09-21T07:26:14.751611+0000","last_active":"2024-09-21T07:26:21.791303+0000","last_peered":"2024-09-21T07:26:21.791303+0000","last_clean":"2024-09-21T07:26:21.791303+0000","last_became_active":"2024-09-21T07:26:14.750246+0000","last_became_peered":"2024-09-21T07:26:14.750246+0000","last_unstale":"2024-09-21T07:26:21.791303+0000","last_undegraded":"2024-09-21T07:26:21.791303+0000","last_fullsized":"2024-09-21T07:26:21.791303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:45.761871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218074+0000","last_change":"2024-09-21T07:26:20.202567+0000","last_active":"2024-09-21T07:26:23.218074+0000","last_peered":"2024-09-21T07:26:23.218074+0000","last_clean":"2024-09-21T07:26:23.218074+0000","last_became_active":"2024-09-21T07:26:20.201637+0000","last_became_peered":"2024-09-21T07:26:20.201637+0000","last_unstale":"2024-09-21T07:26:23.218074+0000","last_undegraded":"2024-09-21T07:26:23.218074+0000","last_fullsized":"2024-09-21T07:26:23.218074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:17.143053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791707+0000","last_change":"2024-09-21T07:26:15.775390+0000","last_active":"2024-09-21T07:26:21.791707+0000","last_peered":"2024-09-21T07:26:21.791707+0000","last_clean":"2024-09-21T07:26:21.791707+0000","last_became_active":"2024-09-21T07:26:15.774972+0000","last_became_peered":"2024-09-21T07:26:15.774972+0000","last_unstale":"2024-09-21T07:26:21.791707+0000","last_undegraded":"2024-09-21T07:26:21.791707+0000","last_fullsized":"2024-09-21T07:26:21.791707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.551786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900819+0000","last_change":"2024-09-21T07:26:14.752809+0000","last_active":"2024-09-21T07:26:22.900819+0000","last_peered":"2024-09-21T07:26:22.900819+0000","last_clean":"2024-09-21T07:26:22.900819+0000","last_became_active":"2024-09-21T07:26:14.752442+0000","last_became_peered":"2024-09-21T07:26:14.752442+0000","last_unstale":"2024-09-21T07:26:22.900819+0000","last_undegraded":"2024-09-21T07:26:22.900819+0000","last_fullsized":"2024-09-21T07:26:22.900819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.370404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218055+0000","last_change":"2024-09-21T07:26:20.202097+0000","last_active":"2024-09-21T07:26:23.218055+0000","last_peered":"2024-09-21T07:26:23.218055+0000","last_clean":"2024-09-21T07:26:23.218055+0000","last_became_active":"2024-09-21T07:26:20.201641+0000","last_became_peered":"2024-09-21T07:26:20.201641+0000","last_unstale":"2024-09-21T07:26:23.218055+0000","last_undegraded":"2024-09-21T07:26:23.218055+0000","last_fullsized":"2024-09-21T07:26:23.218055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:52.838818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755771+0000","last_change":"2024-09-21T07:26:15.763851+0000","last_active":"2024-09-21T07:26:22.755771+0000","last_peered":"2024-09-21T07:26:22.755771+0000","last_clean":"2024-09-21T07:26:22.755771+0000","last_became_active":"2024-09-21T07:26:15.763088+0000","last_became_peered":"2024-09-21T07:26:15.763088+0000","last_unstale":"2024-09-21T07:26:22.755771+0000","last_undegraded":"2024-09-21T07:26:22.755771+0000","last_fullsized":"2024-09-21T07:26:22.755771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:00.411908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756372+0000","last_change":"2024-09-21T07:26:14.749147+0000","last_active":"2024-09-21T07:26:22.756372+0000","last_peered":"2024-09-21T07:26:22.756372+0000","last_clean":"2024-09-21T07:26:22.756372+0000","last_became_active":"2024-09-21T07:26:14.748858+0000","last_became_peered":"2024-09-21T07:26:14.748858+0000","last_unstale":"2024-09-21T07:26:22.756372+0000","last_undegraded":"2024-09-21T07:26:22.756372+0000","last_fullsized":"2024-09-21T07:26:22.756372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:19.994800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901148+0000","last_change":"2024-09-21T07:26:20.202180+0000","last_active":"2024-09-21T07:26:22.901148+0000","last_peered":"2024-09-21T07:26:22.901148+0000","last_clean":"2024-09-21T07:26:22.901148+0000","last_became_active":"2024-09-21T07:26:20.201665+0000","last_became_peered":"2024-09-21T07:26:20.201665+0000","last_unstale":"2024-09-21T07:26:22.901148+0000","last_undegraded":"2024-09-21T07:26:22.901148+0000","last_fullsized":"2024-09-21T07:26:22.901148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:49.726125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.763407+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.763087+0000","last_became_peered":"2024-09-21T07:26:15.763087+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:49.061741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:26.613007+0000","last_change":"2024-09-21T07:26:14.746399+0000","last_active":"2024-09-21T07:26:26.613007+0000","last_peered":"2024-09-21T07:26:26.613007+0000","last_clean":"2024-09-21T07:26:26.613007+0000","last_became_active":"2024-09-21T07:26:14.746029+0000","last_became_peered":"2024-09-21T07:26:14.746029+0000","last_unstale":"2024-09-21T07:26:26.613007+0000","last_undegraded":"2024-09-21T07:26:26.613007+0000","last_fullsized":"2024-09-21T07:26:26.613007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:47.548805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217015+0000","last_change":"2024-09-21T07:26:19.793971+0000","last_active":"2024-09-21T07:26:23.217015+0000","last_peered":"2024-09-21T07:26:23.217015+0000","last_clean":"2024-09-21T07:26:23.217015+0000","last_became_active":"2024-09-21T07:26:19.793342+0000","last_became_peered":"2024-09-21T07:26:19.793342+0000","last_unstale":"2024-09-21T07:26:23.217015+0000","last_undegraded":"2024-09-21T07:26:23.217015+0000","last_fullsized":"2024-09-21T07:26:23.217015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217997+0000","last_change":"2024-09-21T07:26:15.763698+0000","last_active":"2024-09-21T07:26:23.217997+0000","last_peered":"2024-09-21T07:26:23.217997+0000","last_clean":"2024-09-21T07:26:23.217997+0000","last_became_active":"2024-09-21T07:26:15.763010+0000","last_became_peered":"2024-09-21T07:26:15.763010+0000","last_unstale":"2024-09-21T07:26:23.217997+0000","last_undegraded":"2024-09-21T07:26:23.217997+0000","last_fullsized":"2024-09-21T07:26:23.217997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:49.038621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218340+0000","last_change":"2024-09-21T07:26:14.756162+0000","last_active":"2024-09-21T07:26:23.218340+0000","last_peered":"2024-09-21T07:26:23.218340+0000","last_clean":"2024-09-21T07:26:23.218340+0000","last_became_active":"2024-09-21T07:26:14.755821+0000","last_became_peered":"2024-09-21T07:26:14.755821+0000","last_unstale":"2024-09-21T07:26:23.218340+0000","last_undegraded":"2024-09-21T07:26:23.218340+0000","last_fullsized":"2024-09-21T07:26:23.218340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217766+0000","last_change":"2024-09-21T07:26:19.782715+0000","last_active":"2024-09-21T07:26:23.217766+0000","last_peered":"2024-09-21T07:26:23.217766+0000","last_clean":"2024-09-21T07:26:23.217766+0000","last_became_active":"2024-09-21T07:26:19.782285+0000","last_became_peered":"2024-09-21T07:26:19.782285+0000","last_unstale":"2024-09-21T07:26:23.217766+0000","last_undegraded":"2024-09-21T07:26:23.217766+0000","last_fullsized":"2024-09-21T07:26:23.217766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756175+0000","last_change":"2024-09-21T07:26:15.759805+0000","last_active":"2024-09-21T07:26:22.756175+0000","last_peered":"2024-09-21T07:26:22.756175+0000","last_clean":"2024-09-21T07:26:22.756175+0000","last_became_active":"2024-09-21T07:26:15.759487+0000","last_became_peered":"2024-09-21T07:26:15.759487+0000","last_unstale":"2024-09-21T07:26:22.756175+0000","last_undegraded":"2024-09-21T07:26:22.756175+0000","last_fullsized":"2024-09-21T07:26:22.756175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900206+0000","last_change":"2024-09-21T07:26:14.753655+0000","last_active":"2024-09-21T07:26:22.900206+0000","last_peered":"2024-09-21T07:26:22.900206+0000","last_clean":"2024-09-21T07:26:22.900206+0000","last_became_active":"2024-09-21T07:26:14.753266+0000","last_became_peered":"2024-09-21T07:26:14.753266+0000","last_unstale":"2024-09-21T07:26:22.900206+0000","last_undegraded":"2024-09-21T07:26:22.900206+0000","last_fullsized":"2024-09-21T07:26:22.900206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:14.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218754+0000","last_change":"2024-09-21T07:26:19.786292+0000","last_active":"2024-09-21T07:26:23.218754+0000","last_peered":"2024-09-21T07:26:23.218754+0000","last_clean":"2024-09-21T07:26:23.218754+0000","last_became_active":"2024-09-21T07:26:19.783800+0000","last_became_peered":"2024-09-21T07:26:19.783800+0000","last_unstale":"2024-09-21T07:26:23.218754+0000","last_undegraded":"2024-09-21T07:26:23.218754+0000","last_fullsized":"2024-09-21T07:26:23.218754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:52.992488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901112+0000","last_change":"2024-09-21T07:26:15.766957+0000","last_active":"2024-09-21T07:26:22.901112+0000","last_peered":"2024-09-21T07:26:22.901112+0000","last_clean":"2024-09-21T07:26:22.901112+0000","last_became_active":"2024-09-21T07:26:15.766640+0000","last_became_peered":"2024-09-21T07:26:15.766640+0000","last_unstale":"2024-09-21T07:26:22.901112+0000","last_undegraded":"2024-09-21T07:26:22.901112+0000","last_fullsized":"2024-09-21T07:26:22.901112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:23.200622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791558+0000","last_change":"2024-09-21T07:26:14.756852+0000","last_active":"2024-09-21T07:26:21.791558+0000","last_peered":"2024-09-21T07:26:21.791558+0000","last_clean":"2024-09-21T07:26:21.791558+0000","last_became_active":"2024-09-21T07:26:14.756512+0000","last_became_peered":"2024-09-21T07:26:14.756512+0000","last_unstale":"2024-09-21T07:26:21.791558+0000","last_undegraded":"2024-09-21T07:26:21.791558+0000","last_fullsized":"2024-09-21T07:26:21.791558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217003+0000","last_change":"2024-09-21T07:26:20.205385+0000","last_active":"2024-09-21T07:26:23.217003+0000","last_peered":"2024-09-21T07:26:23.217003+0000","last_clean":"2024-09-21T07:26:23.217003+0000","last_became_active":"2024-09-21T07:26:20.205016+0000","last_became_peered":"2024-09-21T07:26:20.205016+0000","last_unstale":"2024-09-21T07:26:23.217003+0000","last_undegraded":"2024-09-21T07:26:23.217003+0000","last_fullsized":"2024-09-21T07:26:23.217003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.612156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755846+0000","last_change":"2024-09-21T07:26:20.202075+0000","last_active":"2024-09-21T07:26:22.755846+0000","last_peered":"2024-09-21T07:26:22.755846+0000","last_clean":"2024-09-21T07:26:22.755846+0000","last_became_active":"2024-09-21T07:26:20.201632+0000","last_became_peered":"2024-09-21T07:26:20.201632+0000","last_unstale":"2024-09-21T07:26:22.755846+0000","last_undegraded":"2024-09-21T07:26:22.755846+0000","last_fullsized":"2024-09-21T07:26:22.755846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216493+0000","last_change":"2024-09-21T07:26:15.762928+0000","last_active":"2024-09-21T07:26:23.216493+0000","last_peered":"2024-09-21T07:26:23.216493+0000","last_clean":"2024-09-21T07:26:23.216493+0000","last_became_active":"2024-09-21T07:26:15.762291+0000","last_became_peered":"2024-09-21T07:26:15.762291+0000","last_unstale":"2024-09-21T07:26:23.216493+0000","last_undegraded":"2024-09-21T07:26:23.216493+0000","last_fullsized":"2024-09-21T07:26:23.216493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900637+0000","last_change":"2024-09-21T07:26:14.755764+0000","last_active":"2024-09-21T07:26:22.900637+0000","last_peered":"2024-09-21T07:26:22.900637+0000","last_clean":"2024-09-21T07:26:22.900637+0000","last_became_active":"2024-09-21T07:26:14.755397+0000","last_became_peered":"2024-09-21T07:26:14.755397+0000","last_unstale":"2024-09-21T07:26:22.900637+0000","last_undegraded":"2024-09-21T07:26:22.900637+0000","last_fullsized":"2024-09-21T07:26:22.900637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:32.946211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791301+0000","last_change":"2024-09-21T07:26:19.789958+0000","last_active":"2024-09-21T07:26:21.791301+0000","last_peered":"2024-09-21T07:26:21.791301+0000","last_clean":"2024-09-21T07:26:21.791301+0000","last_became_active":"2024-09-21T07:26:19.789515+0000","last_became_peered":"2024-09-21T07:26:19.789515+0000","last_unstale":"2024-09-21T07:26:21.791301+0000","last_undegraded":"2024-09-21T07:26:21.791301+0000","last_fullsized":"2024-09-21T07:26:21.791301+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218685+0000","last_change":"2024-09-21T07:26:15.760941+0000","last_active":"2024-09-21T07:26:23.218685+0000","last_peered":"2024-09-21T07:26:23.218685+0000","last_clean":"2024-09-21T07:26:23.218685+0000","last_became_active":"2024-09-21T07:26:15.760612+0000","last_became_peered":"2024-09-21T07:26:15.760612+0000","last_unstale":"2024-09-21T07:26:23.218685+0000","last_undegraded":"2024-09-21T07:26:23.218685+0000","last_fullsized":"2024-09-21T07:26:23.218685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:23.610491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216574+0000","last_change":"2024-09-21T07:26:14.744602+0000","last_active":"2024-09-21T07:26:23.216574+0000","last_peered":"2024-09-21T07:26:23.216574+0000","last_clean":"2024-09-21T07:26:23.216574+0000","last_became_active":"2024-09-21T07:26:14.744284+0000","last_became_peered":"2024-09-21T07:26:14.744284+0000","last_unstale":"2024-09-21T07:26:23.216574+0000","last_undegraded":"2024-09-21T07:26:23.216574+0000","last_fullsized":"2024-09-21T07:26:23.216574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:39.642931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790820+0000","last_change":"2024-09-21T07:26:14.750830+0000","last_active":"2024-09-21T07:26:21.790820+0000","last_peered":"2024-09-21T07:26:21.790820+0000","last_clean":"2024-09-21T07:26:21.790820+0000","last_became_active":"2024-09-21T07:26:14.749815+0000","last_became_peered":"2024-09-21T07:26:14.749815+0000","last_unstale":"2024-09-21T07:26:21.790820+0000","last_undegraded":"2024-09-21T07:26:21.790820+0000","last_fullsized":"2024-09-21T07:26:21.790820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756340+0000","last_change":"2024-09-21T07:26:15.773671+0000","last_active":"2024-09-21T07:26:22.756340+0000","last_peered":"2024-09-21T07:26:22.756340+0000","last_clean":"2024-09-21T07:26:22.756340+0000","last_became_active":"2024-09-21T07:26:15.773380+0000","last_became_peered":"2024-09-21T07:26:15.773380+0000","last_unstale":"2024-09-21T07:26:22.756340+0000","last_undegraded":"2024-09-21T07:26:22.756340+0000","last_fullsized":"2024-09-21T07:26:22.756340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789217+0000","last_change":"2024-09-21T07:26:19.785278+0000","last_active":"2024-09-21T07:26:21.789217+0000","last_peered":"2024-09-21T07:26:21.789217+0000","last_clean":"2024-09-21T07:26:21.789217+0000","last_became_active":"2024-09-21T07:26:19.784142+0000","last_became_peered":"2024-09-21T07:26:19.784142+0000","last_unstale":"2024-09-21T07:26:21.789217+0000","last_undegraded":"2024-09-21T07:26:21.789217+0000","last_fullsized":"2024-09-21T07:26:21.789217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.144434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217614+0000","last_change":"2024-09-21T07:26:14.746012+0000","last_active":"2024-09-21T07:26:23.217614+0000","last_peered":"2024-09-21T07:26:23.217614+0000","last_clean":"2024-09-21T07:26:23.217614+0000","last_became_active":"2024-09-21T07:26:14.745668+0000","last_became_peered":"2024-09-21T07:26:14.745668+0000","last_unstale":"2024-09-21T07:26:23.217614+0000","last_undegraded":"2024-09-21T07:26:23.217614+0000","last_fullsized":"2024-09-21T07:26:23.217614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:03.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217510+0000","last_change":"2024-09-21T07:26:15.764689+0000","last_active":"2024-09-21T07:26:23.217510+0000","last_peered":"2024-09-21T07:26:23.217510+0000","last_clean":"2024-09-21T07:26:23.217510+0000","last_became_active":"2024-09-21T07:26:15.763516+0000","last_became_peered":"2024-09-21T07:26:15.763516+0000","last_unstale":"2024-09-21T07:26:23.217510+0000","last_undegraded":"2024-09-21T07:26:23.217510+0000","last_fullsized":"2024-09-21T07:26:23.217510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.556036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217467+0000","last_change":"2024-09-21T07:26:19.784787+0000","last_active":"2024-09-21T07:26:23.217467+0000","last_peered":"2024-09-21T07:26:23.217467+0000","last_clean":"2024-09-21T07:26:23.217467+0000","last_became_active":"2024-09-21T07:26:19.784262+0000","last_became_peered":"2024-09-21T07:26:19.784262+0000","last_unstale":"2024-09-21T07:26:23.217467+0000","last_undegraded":"2024-09-21T07:26:23.217467+0000","last_fullsized":"2024-09-21T07:26:23.217467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:47.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790131+0000","last_change":"2024-09-21T07:26:15.762178+0000","last_active":"2024-09-21T07:26:21.790131+0000","last_peered":"2024-09-21T07:26:21.790131+0000","last_clean":"2024-09-21T07:26:21.790131+0000","last_became_active":"2024-09-21T07:26:15.761535+0000","last_became_peered":"2024-09-21T07:26:15.761535+0000","last_unstale":"2024-09-21T07:26:21.790131+0000","last_undegraded":"2024-09-21T07:26:21.790131+0000","last_fullsized":"2024-09-21T07:26:21.790131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789216+0000","last_change":"2024-09-21T07:26:15.760862+0000","last_active":"2024-09-21T07:26:21.789216+0000","last_peered":"2024-09-21T07:26:21.789216+0000","last_clean":"2024-09-21T07:26:21.789216+0000","last_became_active":"2024-09-21T07:26:15.760547+0000","last_became_peered":"2024-09-21T07:26:15.760547+0000","last_unstale":"2024-09-21T07:26:21.789216+0000","last_undegraded":"2024-09-21T07:26:21.789216+0000","last_fullsized":"2024-09-21T07:26:21.789216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:49.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900702+0000","last_change":"2024-09-21T07:26:15.773293+0000","last_active":"2024-09-21T07:26:22.900702+0000","last_peered":"2024-09-21T07:26:22.900702+0000","last_clean":"2024-09-21T07:26:22.900702+0000","last_became_active":"2024-09-21T07:26:15.772904+0000","last_became_peered":"2024-09-21T07:26:15.772904+0000","last_unstale":"2024-09-21T07:26:22.900702+0000","last_undegraded":"2024-09-21T07:26:22.900702+0000","last_fullsized":"2024-09-21T07:26:22.900702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:56.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756234+0000","last_change":"2024-09-21T07:26:15.764081+0000","last_active":"2024-09-21T07:26:22.756234+0000","last_peered":"2024-09-21T07:26:22.756234+0000","last_clean":"2024-09-21T07:26:22.756234+0000","last_became_active":"2024-09-21T07:26:15.763052+0000","last_became_peered":"2024-09-21T07:26:15.763052+0000","last_unstale":"2024-09-21T07:26:22.756234+0000","last_undegraded":"2024-09-21T07:26:22.756234+0000","last_fullsized":"2024-09-21T07:26:22.756234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218342+0000","last_change":"2024-09-21T07:26:15.766994+0000","last_active":"2024-09-21T07:26:23.218342+0000","last_peered":"2024-09-21T07:26:23.218342+0000","last_clean":"2024-09-21T07:26:23.218342+0000","last_became_active":"2024-09-21T07:26:15.766623+0000","last_became_peered":"2024-09-21T07:26:15.766623+0000","last_unstale":"2024-09-21T07:26:23.218342+0000","last_undegraded":"2024-09-21T07:26:23.218342+0000","last_fullsized":"2024-09-21T07:26:23.218342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:59.716664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218240+0000","last_change":"2024-09-21T07:26:15.758623+0000","last_active":"2024-09-21T07:26:23.218240+0000","last_peered":"2024-09-21T07:26:23.218240+0000","last_clean":"2024-09-21T07:26:23.218240+0000","last_became_active":"2024-09-21T07:26:15.758309+0000","last_became_peered":"2024-09-21T07:26:15.758309+0000","last_unstale":"2024-09-21T07:26:23.218240+0000","last_undegraded":"2024-09-21T07:26:23.218240+0000","last_fullsized":"2024-09-21T07:26:23.218240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792006+0000","last_change":"2024-09-21T07:26:15.770009+0000","last_active":"2024-09-21T07:26:21.792006+0000","last_peered":"2024-09-21T07:26:21.792006+0000","last_clean":"2024-09-21T07:26:21.792006+0000","last_became_active":"2024-09-21T07:26:15.769549+0000","last_became_peered":"2024-09-21T07:26:15.769549+0000","last_unstale":"2024-09-21T07:26:21.792006+0000","last_undegraded":"2024-09-21T07:26:21.792006+0000","last_fullsized":"2024-09-21T07:26:21.792006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218223+0000","last_change":"2024-09-21T07:26:15.765956+0000","last_active":"2024-09-21T07:26:23.218223+0000","last_peered":"2024-09-21T07:26:23.218223+0000","last_clean":"2024-09-21T07:26:23.218223+0000","last_became_active":"2024-09-21T07:26:15.765561+0000","last_became_peered":"2024-09-21T07:26:15.765561+0000","last_unstale":"2024-09-21T07:26:23.218223+0000","last_undegraded":"2024-09-21T07:26:23.218223+0000","last_fullsized":"2024-09-21T07:26:23.218223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:35.944784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900539+0000","last_change":"2024-09-21T07:26:15.766091+0000","last_active":"2024-09-21T07:26:22.900539+0000","last_peered":"2024-09-21T07:26:22.900539+0000","last_clean":"2024-09-21T07:26:22.900539+0000","last_became_active":"2024-09-21T07:26:15.765771+0000","last_became_peered":"2024-09-21T07:26:15.765771+0000","last_unstale":"2024-09-21T07:26:22.900539+0000","last_undegraded":"2024-09-21T07:26:22.900539+0000","last_fullsized":"2024-09-21T07:26:22.900539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:15.463622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217240+0000","last_change":"2024-09-21T07:26:15.771905+0000","last_active":"2024-09-21T07:26:23.217240+0000","last_peered":"2024-09-21T07:26:23.217240+0000","last_clean":"2024-09-21T07:26:23.217240+0000","last_became_active":"2024-09-21T07:26:15.771525+0000","last_became_peered":"2024-09-21T07:26:15.771525+0000","last_unstale":"2024-09-21T07:26:23.217240+0000","last_undegraded":"2024-09-21T07:26:23.217240+0000","last_fullsized":"2024-09-21T07:26:23.217240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.988043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900725+0000","last_change":"2024-09-21T07:26:15.775865+0000","last_active":"2024-09-21T07:26:22.900725+0000","last_peered":"2024-09-21T07:26:22.900725+0000","last_clean":"2024-09-21T07:26:22.900725+0000","last_became_active":"2024-09-21T07:26:15.773173+0000","last_became_peered":"2024-09-21T07:26:15.773173+0000","last_unstale":"2024-09-21T07:26:22.900725+0000","last_undegraded":"2024-09-21T07:26:22.900725+0000","last_fullsized":"2024-09-21T07:26:22.900725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:42.898368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756041+0000","last_change":"2024-09-21T07:26:15.766057+0000","last_active":"2024-09-21T07:26:22.756041+0000","last_peered":"2024-09-21T07:26:22.756041+0000","last_clean":"2024-09-21T07:26:22.756041+0000","last_became_active":"2024-09-21T07:26:15.765714+0000","last_became_peered":"2024-09-21T07:26:15.765714+0000","last_unstale":"2024-09-21T07:26:22.756041+0000","last_undegraded":"2024-09-21T07:26:22.756041+0000","last_fullsized":"2024-09-21T07:26:22.756041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790549+0000","last_change":"2024-09-21T07:26:15.774046+0000","last_active":"2024-09-21T07:26:21.790549+0000","last_peered":"2024-09-21T07:26:21.790549+0000","last_clean":"2024-09-21T07:26:21.790549+0000","last_became_active":"2024-09-21T07:26:15.773709+0000","last_became_peered":"2024-09-21T07:26:15.773709+0000","last_unstale":"2024-09-21T07:26:21.790549+0000","last_undegraded":"2024-09-21T07:26:21.790549+0000","last_fullsized":"2024-09-21T07:26:21.790549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791898+0000","last_change":"2024-09-21T07:26:15.760349+0000","last_active":"2024-09-21T07:26:21.791898+0000","last_peered":"2024-09-21T07:26:21.791898+0000","last_clean":"2024-09-21T07:26:21.791898+0000","last_became_active":"2024-09-21T07:26:15.759759+0000","last_became_peered":"2024-09-21T07:26:15.759759+0000","last_unstale":"2024-09-21T07:26:21.791898+0000","last_undegraded":"2024-09-21T07:26:21.791898+0000","last_fullsized":"2024-09-21T07:26:21.791898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:33.433481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789376+0000","last_change":"2024-09-21T07:26:15.767539+0000","last_active":"2024-09-21T07:26:21.789376+0000","last_peered":"2024-09-21T07:26:21.789376+0000","last_clean":"2024-09-21T07:26:21.789376+0000","last_became_active":"2024-09-21T07:26:15.767174+0000","last_became_peered":"2024-09-21T07:26:15.767174+0000","last_unstale":"2024-09-21T07:26:21.789376+0000","last_undegraded":"2024-09-21T07:26:21.789376+0000","last_fullsized":"2024-09-21T07:26:21.789376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:08.568390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901369+0000","last_change":"2024-09-21T07:26:15.767788+0000","last_active":"2024-09-21T07:26:22.901369+0000","last_peered":"2024-09-21T07:26:22.901369+0000","last_clean":"2024-09-21T07:26:22.901369+0000","last_became_active":"2024-09-21T07:26:15.767459+0000","last_became_peered":"2024-09-21T07:26:15.767459+0000","last_unstale":"2024-09-21T07:26:22.901369+0000","last_undegraded":"2024-09-21T07:26:22.901369+0000","last_fullsized":"2024-09-21T07:26:22.901369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.318947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900393+0000","last_change":"2024-09-21T07:26:15.758591+0000","last_active":"2024-09-21T07:26:22.900393+0000","last_peered":"2024-09-21T07:26:22.900393+0000","last_clean":"2024-09-21T07:26:22.900393+0000","last_became_active":"2024-09-21T07:26:15.758248+0000","last_became_peered":"2024-09-21T07:26:15.758248+0000","last_unstale":"2024-09-21T07:26:22.900393+0000","last_undegraded":"2024-09-21T07:26:22.900393+0000","last_fullsized":"2024-09-21T07:26:22.900393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:36.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216601+0000","last_change":"2024-09-21T07:26:15.763665+0000","last_active":"2024-09-21T07:26:23.216601+0000","last_peered":"2024-09-21T07:26:23.216601+0000","last_clean":"2024-09-21T07:26:23.216601+0000","last_became_active":"2024-09-21T07:26:15.763348+0000","last_became_peered":"2024-09-21T07:26:15.763348+0000","last_unstale":"2024-09-21T07:26:23.216601+0000","last_undegraded":"2024-09-21T07:26:23.216601+0000","last_fullsized":"2024-09-21T07:26:23.216601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755781+0000","last_change":"2024-09-21T07:26:15.757427+0000","last_active":"2024-09-21T07:26:22.755781+0000","last_peered":"2024-09-21T07:26:22.755781+0000","last_clean":"2024-09-21T07:26:22.755781+0000","last_became_active":"2024-09-21T07:26:15.757078+0000","last_became_peered":"2024-09-21T07:26:15.757078+0000","last_unstale":"2024-09-21T07:26:22.755781+0000","last_undegraded":"2024-09-21T07:26:22.755781+0000","last_fullsized":"2024-09-21T07:26:22.755781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:46.647829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:15.767064+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:15.766690+0000","last_became_peered":"2024-09-21T07:26:15.766690+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:07.860716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217825+0000","last_change":"2024-09-21T07:26:15.775975+0000","last_active":"2024-09-21T07:26:23.217825+0000","last_peered":"2024-09-21T07:26:23.217825+0000","last_clean":"2024-09-21T07:26:23.217825+0000","last_became_active":"2024-09-21T07:26:15.775526+0000","last_became_peered":"2024-09-21T07:26:15.775526+0000","last_unstale":"2024-09-21T07:26:23.217825+0000","last_undegraded":"2024-09-21T07:26:23.217825+0000","last_fullsized":"2024-09-21T07:26:23.217825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.840733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791070+0000","last_change":"2024-09-21T07:26:15.776942+0000","last_active":"2024-09-21T07:26:21.791070+0000","last_peered":"2024-09-21T07:26:21.791070+0000","last_clean":"2024-09-21T07:26:21.791070+0000","last_became_active":"2024-09-21T07:26:15.776632+0000","last_became_peered":"2024-09-21T07:26:15.776632+0000","last_unstale":"2024-09-21T07:26:21.791070+0000","last_undegraded":"2024-09-21T07:26:21.791070+0000","last_fullsized":"2024-09-21T07:26:21.791070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791878+0000","last_change":"2024-09-21T07:26:15.766327+0000","last_active":"2024-09-21T07:26:21.791878+0000","last_peered":"2024-09-21T07:26:21.791878+0000","last_clean":"2024-09-21T07:26:21.791878+0000","last_became_active":"2024-09-21T07:26:15.765981+0000","last_became_peered":"2024-09-21T07:26:15.765981+0000","last_unstale":"2024-09-21T07:26:21.791878+0000","last_undegraded":"2024-09-21T07:26:21.791878+0000","last_fullsized":"2024-09-21T07:26:21.791878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791919+0000","last_change":"2024-09-21T07:26:15.777412+0000","last_active":"2024-09-21T07:26:21.791919+0000","last_peered":"2024-09-21T07:26:21.791919+0000","last_clean":"2024-09-21T07:26:21.791919+0000","last_became_active":"2024-09-21T07:26:15.776925+0000","last_became_peered":"2024-09-21T07:26:15.776925+0000","last_unstale":"2024-09-21T07:26:21.791919+0000","last_undegraded":"2024-09-21T07:26:21.791919+0000","last_fullsized":"2024-09-21T07:26:21.791919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:38.270679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218556+0000","last_change":"2024-09-21T07:26:15.777075+0000","last_active":"2024-09-21T07:26:23.218556+0000","last_peered":"2024-09-21T07:26:23.218556+0000","last_clean":"2024-09-21T07:26:23.218556+0000","last_became_active":"2024-09-21T07:26:15.776611+0000","last_became_peered":"2024-09-21T07:26:15.776611+0000","last_unstale":"2024-09-21T07:26:23.218556+0000","last_undegraded":"2024-09-21T07:26:23.218556+0000","last_fullsized":"2024-09-21T07:26:23.218556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:38.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790866+0000","last_change":"2024-09-21T07:26:15.762408+0000","last_active":"2024-09-21T07:26:21.790866+0000","last_peered":"2024-09-21T07:26:21.790866+0000","last_clean":"2024-09-21T07:26:21.790866+0000","last_became_active":"2024-09-21T07:26:15.762014+0000","last_became_peered":"2024-09-21T07:26:15.762014+0000","last_unstale":"2024-09-21T07:26:21.790866+0000","last_undegraded":"2024-09-21T07:26:21.790866+0000","last_fullsized":"2024-09-21T07:26:21.790866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:09.186226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900571+0000","last_change":"2024-09-21T07:26:15.759759+0000","last_active":"2024-09-21T07:26:22.900571+0000","last_peered":"2024-09-21T07:26:22.900571+0000","last_clean":"2024-09-21T07:26:22.900571+0000","last_became_active":"2024-09-21T07:26:15.758871+0000","last_became_peered":"2024-09-21T07:26:15.758871+0000","last_unstale":"2024-09-21T07:26:22.900571+0000","last_undegraded":"2024-09-21T07:26:22.900571+0000","last_fullsized":"2024-09-21T07:26:22.900571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:23.666201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791461+0000","last_change":"2024-09-21T07:26:15.768245+0000","last_active":"2024-09-21T07:26:21.791461+0000","last_peered":"2024-09-21T07:26:21.791461+0000","last_clean":"2024-09-21T07:26:21.791461+0000","last_became_active":"2024-09-21T07:26:15.767901+0000","last_became_peered":"2024-09-21T07:26:15.767901+0000","last_unstale":"2024-09-21T07:26:21.791461+0000","last_undegraded":"2024-09-21T07:26:21.791461+0000","last_fullsized":"2024-09-21T07:26:21.791461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:52.823399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900922+0000","last_change":"2024-09-21T07:26:15.772040+0000","last_active":"2024-09-21T07:26:22.900922+0000","last_peered":"2024-09-21T07:26:22.900922+0000","last_clean":"2024-09-21T07:26:22.900922+0000","last_became_active":"2024-09-21T07:26:15.771194+0000","last_became_peered":"2024-09-21T07:26:15.771194+0000","last_unstale":"2024-09-21T07:26:22.900922+0000","last_undegraded":"2024-09-21T07:26:22.900922+0000","last_fullsized":"2024-09-21T07:26:22.900922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:35.969051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755834+0000","last_change":"2024-09-21T07:26:15.752787+0000","last_active":"2024-09-21T07:26:22.755834+0000","last_peered":"2024-09-21T07:26:22.755834+0000","last_clean":"2024-09-21T07:26:22.755834+0000","last_became_active":"2024-09-21T07:26:15.752457+0000","last_became_peered":"2024-09-21T07:26:15.752457+0000","last_unstale":"2024-09-21T07:26:22.755834+0000","last_undegraded":"2024-09-21T07:26:22.755834+0000","last_fullsized":"2024-09-21T07:26:22.755834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217328+0000","last_change":"2024-09-21T07:26:15.754185+0000","last_active":"2024-09-21T07:26:23.217328+0000","last_peered":"2024-09-21T07:26:23.217328+0000","last_clean":"2024-09-21T07:26:23.217328+0000","last_became_active":"2024-09-21T07:26:15.753790+0000","last_became_peered":"2024-09-21T07:26:15.753790+0000","last_unstale":"2024-09-21T07:26:23.217328+0000","last_undegraded":"2024-09-21T07:26:23.217328+0000","last_fullsized":"2024-09-21T07:26:23.217328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.769408+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.767688+0000","last_became_peered":"2024-09-21T07:26:15.767688+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:46.925801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":704845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":113985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":114021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":113895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":114055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":888832,"data_stored":704901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:26:36.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:26:36.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 -- 172.21.15.105:0/240727248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c80b6850 msgr2=0x7f09c80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 --2- 172.21.15.105:0/240727248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c80b6850 0x7f09c80cf590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f09c0005fd0 tx=0x7f09c002a820 comp rx=0 tx=0).stop 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 -- 172.21.15.105:0/240727248 shutdown_connections 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 --2- 172.21.15.105:0/240727248 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c80b6850 0x7f09c80cf590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 --2- 172.21.15.105:0/240727248 >> v2:172.21.15.112:3300/0 conn(0x7f09c80b6440 0x7f09c80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 -- 172.21.15.105:0/240727248 >> 172.21.15.105:0/240727248 conn(0x7f09c808bfb0 msgr2=0x7f09c80d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 -- 172.21.15.105:0/240727248 shutdown_connections 2024-09-21T07:26:36.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.980+0000 7f09ccab4640 1 -- 172.21.15.105:0/240727248 wait complete. 2024-09-21T07:26:36.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.982+0000 7f09ccab4640 1 Processor -- start 2024-09-21T07:26:36.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.982+0000 7f09ccab4640 1 -- start start 2024-09-21T07:26:36.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c808c3c0 0x7f09c81f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c80b6850 0x7f09c80ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80cfce0 con 0x7f09c80b6440 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f09c80b3b40 con 0x7f09c80b6850 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f09c8162c70 con 0x7f09c808c3c0 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59656/0 (socket says 172.21.15.105:59656) 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 -- 172.21.15.105:0/1115290698 learned_addr learned my addr 172.21.15.105:0/1115290698 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c659c640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c808c3c0 0x7f09c81f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c6d9d640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c80b6850 0x7f09c80ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 -- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c808c3c0 msgr2=0x7f09c81f4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c808c3c0 0x7f09c81f4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 -- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c80b6850 msgr2=0x7f09c80ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c80b6850 0x7f09c80ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 -- 172.21.15.105:0/1115290698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c8151a10 con 0x7f09c80b6440 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09c5d9b640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00ea20 tx=0x7f09bc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00cd20 con 0x7f09c80b6440 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09bc010430 con 0x7f09c80b6440 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c8161d40 con 0x7f09c80b6440 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc010860 con 0x7f09c80b6440 2024-09-21T07:26:36.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.984+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81622e0 con 0x7f09c80b6440 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.985+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c806f7f0 con 0x7f09c80b6440 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.986+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09bc010a00 con 0x7f09c80b6440 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.987+0000 7f09b6ffd640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 0x7f09a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.987+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f09bc09b940 con 0x7f09c80b6440 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.987+0000 7f09c659c640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 0x7f09a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:36.989 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.987+0000 7f09c659c640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 0x7f09a807b0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09b8001720 tx=0x7f09b8007e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:36.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:36.991+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09bc063400 con 0x7f09c80b6440 2024-09-21T07:26:37.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.165+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f09c815fc70 con 0x7f09a8078bc0 2024-09-21T07:26:37.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.171+0000 7f09b6ffd640 1 -- 172.21.15.105:0/1115290698 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f09b8001d00 con 0x7f09a8078bc0 2024-09-21T07:26:37.176 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:37.178 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 msgr2=0x7f09a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 0x7f09a807b0d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f09b8001720 tx=0x7f09b8007e90 comp rx=0 tx=0).stop 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 msgr2=0x7f09c81f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00ea20 tx=0x7f09bc00eee0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 shutdown_connections 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f09a8078bc0 0x7f09a807b0d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f09c80b6850 0x7f09c80ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f09c80b6440 0x7f09c81f54d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 --2- 172.21.15.105:0/1115290698 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09c808c3c0 0x7f09c81f4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 >> 172.21.15.105:0/1115290698 conn(0x7f09c808bfb0 msgr2=0x7f09c80b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 shutdown_connections 2024-09-21T07:26:37.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.174+0000 7f09ccab4640 1 -- 172.21.15.105:0/1115290698 wait complete. 2024-09-21T07:26:37.187 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-21T07:26:36.770989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3514368,"data_stored":2101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003728"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755382+0000","last_change":"2024-09-21T07:26:14.746008+0000","last_active":"2024-09-21T07:26:22.755382+0000","last_peered":"2024-09-21T07:26:22.755382+0000","last_clean":"2024-09-21T07:26:22.755382+0000","last_became_active":"2024-09-21T07:26:14.745685+0000","last_became_peered":"2024-09-21T07:26:14.745685+0000","last_unstale":"2024-09-21T07:26:22.755382+0000","last_undegraded":"2024-09-21T07:26:22.755382+0000","last_fullsized":"2024-09-21T07:26:22.755382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:02.797953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217709+0000","last_change":"2024-09-21T07:26:15.767737+0000","last_active":"2024-09-21T07:26:23.217709+0000","last_peered":"2024-09-21T07:26:23.217709+0000","last_clean":"2024-09-21T07:26:23.217709+0000","last_became_active":"2024-09-21T07:26:15.767112+0000","last_became_peered":"2024-09-21T07:26:15.767112+0000","last_unstale":"2024-09-21T07:26:23.217709+0000","last_undegraded":"2024-09-21T07:26:23.217709+0000","last_fullsized":"2024-09-21T07:26:23.217709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.781454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217413+0000","last_change":"2024-09-21T07:26:20.205421+0000","last_active":"2024-09-21T07:26:23.217413+0000","last_peered":"2024-09-21T07:26:23.217413+0000","last_clean":"2024-09-21T07:26:23.217413+0000","last_became_active":"2024-09-21T07:26:20.204943+0000","last_became_peered":"2024-09-21T07:26:20.204943+0000","last_unstale":"2024-09-21T07:26:23.217413+0000","last_undegraded":"2024-09-21T07:26:23.217413+0000","last_fullsized":"2024-09-21T07:26:23.217413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:55.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:14.738042+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:14.737714+0000","last_became_peered":"2024-09-21T07:26:14.737714+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:43.999074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756065+0000","last_change":"2024-09-21T07:26:15.760507+0000","last_active":"2024-09-21T07:26:22.756065+0000","last_peered":"2024-09-21T07:26:22.756065+0000","last_clean":"2024-09-21T07:26:22.756065+0000","last_became_active":"2024-09-21T07:26:15.759859+0000","last_became_peered":"2024-09-21T07:26:15.759859+0000","last_unstale":"2024-09-21T07:26:22.756065+0000","last_undegraded":"2024-09-21T07:26:22.756065+0000","last_fullsized":"2024-09-21T07:26:22.756065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217096+0000","last_change":"2024-09-21T07:26:19.783194+0000","last_active":"2024-09-21T07:26:23.217096+0000","last_peered":"2024-09-21T07:26:23.217096+0000","last_clean":"2024-09-21T07:26:23.217096+0000","last_became_active":"2024-09-21T07:26:19.782625+0000","last_became_peered":"2024-09-21T07:26:19.782625+0000","last_unstale":"2024-09-21T07:26:23.217096+0000","last_undegraded":"2024-09-21T07:26:23.217096+0000","last_fullsized":"2024-09-21T07:26:23.217096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791768+0000","last_change":"2024-09-21T07:26:14.748310+0000","last_active":"2024-09-21T07:26:21.791768+0000","last_peered":"2024-09-21T07:26:21.791768+0000","last_clean":"2024-09-21T07:26:21.791768+0000","last_became_active":"2024-09-21T07:26:14.747943+0000","last_became_peered":"2024-09-21T07:26:14.747943+0000","last_unstale":"2024-09-21T07:26:21.791768+0000","last_undegraded":"2024-09-21T07:26:21.791768+0000","last_fullsized":"2024-09-21T07:26:21.791768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:12.833502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901017+0000","last_change":"2024-09-21T07:26:15.772109+0000","last_active":"2024-09-21T07:26:22.901017+0000","last_peered":"2024-09-21T07:26:22.901017+0000","last_clean":"2024-09-21T07:26:22.901017+0000","last_became_active":"2024-09-21T07:26:15.771762+0000","last_became_peered":"2024-09-21T07:26:15.771762+0000","last_unstale":"2024-09-21T07:26:22.901017+0000","last_undegraded":"2024-09-21T07:26:22.901017+0000","last_fullsized":"2024-09-21T07:26:22.901017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:14.024694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217587+0000","last_change":"2024-09-21T07:26:19.782636+0000","last_active":"2024-09-21T07:26:23.217587+0000","last_peered":"2024-09-21T07:26:23.217587+0000","last_clean":"2024-09-21T07:26:23.217587+0000","last_became_active":"2024-09-21T07:26:19.782148+0000","last_became_peered":"2024-09-21T07:26:19.782148+0000","last_unstale":"2024-09-21T07:26:23.217587+0000","last_undegraded":"2024-09-21T07:26:23.217587+0000","last_fullsized":"2024-09-21T07:26:23.217587+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:34.102740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791003+0000","last_change":"2024-09-21T07:26:14.756864+0000","last_active":"2024-09-21T07:26:21.791003+0000","last_peered":"2024-09-21T07:26:21.791003+0000","last_clean":"2024-09-21T07:26:21.791003+0000","last_became_active":"2024-09-21T07:26:14.756519+0000","last_became_peered":"2024-09-21T07:26:14.756519+0000","last_unstale":"2024-09-21T07:26:21.791003+0000","last_undegraded":"2024-09-21T07:26:21.791003+0000","last_fullsized":"2024-09-21T07:26:21.791003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.306611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901081+0000","last_change":"2024-09-21T07:26:15.769890+0000","last_active":"2024-09-21T07:26:22.901081+0000","last_peered":"2024-09-21T07:26:22.901081+0000","last_clean":"2024-09-21T07:26:22.901081+0000","last_became_active":"2024-09-21T07:26:15.769124+0000","last_became_peered":"2024-09-21T07:26:15.769124+0000","last_unstale":"2024-09-21T07:26:22.901081+0000","last_undegraded":"2024-09-21T07:26:22.901081+0000","last_fullsized":"2024-09-21T07:26:22.901081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:55.122043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217329+0000","last_change":"2024-09-21T07:26:19.788443+0000","last_active":"2024-09-21T07:26:23.217329+0000","last_peered":"2024-09-21T07:26:23.217329+0000","last_clean":"2024-09-21T07:26:23.217329+0000","last_became_active":"2024-09-21T07:26:19.787922+0000","last_became_peered":"2024-09-21T07:26:19.787922+0000","last_unstale":"2024-09-21T07:26:23.217329+0000","last_undegraded":"2024-09-21T07:26:23.217329+0000","last_fullsized":"2024-09-21T07:26:23.217329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790557+0000","last_change":"2024-09-21T07:26:15.774537+0000","last_active":"2024-09-21T07:26:21.790557+0000","last_peered":"2024-09-21T07:26:21.790557+0000","last_clean":"2024-09-21T07:26:21.790557+0000","last_became_active":"2024-09-21T07:26:15.774215+0000","last_became_peered":"2024-09-21T07:26:15.774215+0000","last_unstale":"2024-09-21T07:26:21.790557+0000","last_undegraded":"2024-09-21T07:26:21.790557+0000","last_fullsized":"2024-09-21T07:26:21.790557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791990+0000","last_change":"2024-09-21T07:26:14.752255+0000","last_active":"2024-09-21T07:26:21.791990+0000","last_peered":"2024-09-21T07:26:21.791990+0000","last_clean":"2024-09-21T07:26:21.791990+0000","last_became_active":"2024-09-21T07:26:14.751809+0000","last_became_peered":"2024-09-21T07:26:14.751809+0000","last_unstale":"2024-09-21T07:26:21.791990+0000","last_undegraded":"2024-09-21T07:26:21.791990+0000","last_fullsized":"2024-09-21T07:26:21.791990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:30.055064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218154+0000","last_change":"2024-09-21T07:26:19.784193+0000","last_active":"2024-09-21T07:26:23.218154+0000","last_peered":"2024-09-21T07:26:23.218154+0000","last_clean":"2024-09-21T07:26:23.218154+0000","last_became_active":"2024-09-21T07:26:19.783666+0000","last_became_peered":"2024-09-21T07:26:19.783666+0000","last_unstale":"2024-09-21T07:26:23.218154+0000","last_undegraded":"2024-09-21T07:26:23.218154+0000","last_fullsized":"2024-09-21T07:26:23.218154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218051+0000","last_change":"2024-09-21T07:26:15.761293+0000","last_active":"2024-09-21T07:26:23.218051+0000","last_peered":"2024-09-21T07:26:23.218051+0000","last_clean":"2024-09-21T07:26:23.218051+0000","last_became_active":"2024-09-21T07:26:15.760959+0000","last_became_peered":"2024-09-21T07:26:15.760959+0000","last_unstale":"2024-09-21T07:26:23.218051+0000","last_undegraded":"2024-09-21T07:26:23.218051+0000","last_fullsized":"2024-09-21T07:26:23.218051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:12.766792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218529+0000","last_change":"2024-09-21T07:26:14.737676+0000","last_active":"2024-09-21T07:26:23.218529+0000","last_peered":"2024-09-21T07:26:23.218529+0000","last_clean":"2024-09-21T07:26:23.218529+0000","last_became_active":"2024-09-21T07:26:14.737367+0000","last_became_peered":"2024-09-21T07:26:14.737367+0000","last_unstale":"2024-09-21T07:26:23.218529+0000","last_undegraded":"2024-09-21T07:26:23.218529+0000","last_fullsized":"2024-09-21T07:26:23.218529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217727+0000","last_change":"2024-09-21T07:26:19.782689+0000","last_active":"2024-09-21T07:26:23.217727+0000","last_peered":"2024-09-21T07:26:23.217727+0000","last_clean":"2024-09-21T07:26:23.217727+0000","last_became_active":"2024-09-21T07:26:19.782215+0000","last_became_peered":"2024-09-21T07:26:19.782215+0000","last_unstale":"2024-09-21T07:26:23.217727+0000","last_undegraded":"2024-09-21T07:26:23.217727+0000","last_fullsized":"2024-09-21T07:26:23.217727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218798+0000","last_change":"2024-09-21T07:26:15.758262+0000","last_active":"2024-09-21T07:26:23.218798+0000","last_peered":"2024-09-21T07:26:23.218798+0000","last_clean":"2024-09-21T07:26:23.218798+0000","last_became_active":"2024-09-21T07:26:15.757944+0000","last_became_peered":"2024-09-21T07:26:15.757944+0000","last_unstale":"2024-09-21T07:26:23.218798+0000","last_undegraded":"2024-09-21T07:26:23.218798+0000","last_fullsized":"2024-09-21T07:26:23.218798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:10.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216341+0000","last_change":"2024-09-21T07:26:14.745239+0000","last_active":"2024-09-21T07:26:23.216341+0000","last_peered":"2024-09-21T07:26:23.216341+0000","last_clean":"2024-09-21T07:26:23.216341+0000","last_became_active":"2024-09-21T07:26:14.744901+0000","last_became_peered":"2024-09-21T07:26:14.744901+0000","last_unstale":"2024-09-21T07:26:23.216341+0000","last_undegraded":"2024-09-21T07:26:23.216341+0000","last_fullsized":"2024-09-21T07:26:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:07.561158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218435+0000","last_change":"2024-09-21T07:26:20.205294+0000","last_active":"2024-09-21T07:26:23.218435+0000","last_peered":"2024-09-21T07:26:23.218435+0000","last_clean":"2024-09-21T07:26:23.218435+0000","last_became_active":"2024-09-21T07:26:20.204855+0000","last_became_peered":"2024-09-21T07:26:20.204855+0000","last_unstale":"2024-09-21T07:26:23.218435+0000","last_undegraded":"2024-09-21T07:26:23.218435+0000","last_fullsized":"2024-09-21T07:26:23.218435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217428+0000","last_change":"2024-09-21T07:26:15.762544+0000","last_active":"2024-09-21T07:26:23.217428+0000","last_peered":"2024-09-21T07:26:23.217428+0000","last_clean":"2024-09-21T07:26:23.217428+0000","last_became_active":"2024-09-21T07:26:15.762193+0000","last_became_peered":"2024-09-21T07:26:15.762193+0000","last_unstale":"2024-09-21T07:26:23.217428+0000","last_undegraded":"2024-09-21T07:26:23.217428+0000","last_fullsized":"2024-09-21T07:26:23.217428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:42.681607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790143+0000","last_change":"2024-09-21T07:26:14.748084+0000","last_active":"2024-09-21T07:26:21.790143+0000","last_peered":"2024-09-21T07:26:21.790143+0000","last_clean":"2024-09-21T07:26:21.790143+0000","last_became_active":"2024-09-21T07:26:14.747288+0000","last_became_peered":"2024-09-21T07:26:14.747288+0000","last_unstale":"2024-09-21T07:26:21.790143+0000","last_undegraded":"2024-09-21T07:26:21.790143+0000","last_fullsized":"2024-09-21T07:26:21.790143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:39.950583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:19.788640+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:19.788072+0000","last_became_peered":"2024-09-21T07:26:19.788072+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756286+0000","last_change":"2024-09-21T07:26:15.757686+0000","last_active":"2024-09-21T07:26:22.756286+0000","last_peered":"2024-09-21T07:26:22.756286+0000","last_clean":"2024-09-21T07:26:22.756286+0000","last_became_active":"2024-09-21T07:26:15.757361+0000","last_became_peered":"2024-09-21T07:26:15.757361+0000","last_unstale":"2024-09-21T07:26:22.756286+0000","last_undegraded":"2024-09-21T07:26:22.756286+0000","last_fullsized":"2024-09-21T07:26:22.756286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:49.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791712+0000","last_change":"2024-09-21T07:26:14.746130+0000","last_active":"2024-09-21T07:26:21.791712+0000","last_peered":"2024-09-21T07:26:21.791712+0000","last_clean":"2024-09-21T07:26:21.791712+0000","last_became_active":"2024-09-21T07:26:14.745763+0000","last_became_peered":"2024-09-21T07:26:14.745763+0000","last_unstale":"2024-09-21T07:26:21.791712+0000","last_undegraded":"2024-09-21T07:26:21.791712+0000","last_fullsized":"2024-09-21T07:26:21.791712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217052+0000","last_change":"2024-09-21T07:26:20.203703+0000","last_active":"2024-09-21T07:26:23.217052+0000","last_peered":"2024-09-21T07:26:23.217052+0000","last_clean":"2024-09-21T07:26:23.217052+0000","last_became_active":"2024-09-21T07:26:20.202561+0000","last_became_peered":"2024-09-21T07:26:20.202561+0000","last_unstale":"2024-09-21T07:26:23.217052+0000","last_undegraded":"2024-09-21T07:26:23.217052+0000","last_fullsized":"2024-09-21T07:26:23.217052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:27.041439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218313+0000","last_change":"2024-09-21T07:26:15.768378+0000","last_active":"2024-09-21T07:26:23.218313+0000","last_peered":"2024-09-21T07:26:23.218313+0000","last_clean":"2024-09-21T07:26:23.218313+0000","last_became_active":"2024-09-21T07:26:15.768068+0000","last_became_peered":"2024-09-21T07:26:15.768068+0000","last_unstale":"2024-09-21T07:26:23.218313+0000","last_undegraded":"2024-09-21T07:26:23.218313+0000","last_fullsized":"2024-09-21T07:26:23.218313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:12.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216461+0000","last_change":"2024-09-21T07:26:14.741727+0000","last_active":"2024-09-21T07:26:23.216461+0000","last_peered":"2024-09-21T07:26:23.216461+0000","last_clean":"2024-09-21T07:26:23.216461+0000","last_became_active":"2024-09-21T07:26:14.741372+0000","last_became_peered":"2024-09-21T07:26:14.741372+0000","last_unstale":"2024-09-21T07:26:23.216461+0000","last_undegraded":"2024-09-21T07:26:23.216461+0000","last_fullsized":"2024-09-21T07:26:23.216461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:48.123776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218506+0000","last_change":"2024-09-21T07:26:20.203003+0000","last_active":"2024-09-21T07:26:23.218506+0000","last_peered":"2024-09-21T07:26:23.218506+0000","last_clean":"2024-09-21T07:26:23.218506+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:23.218506+0000","last_undegraded":"2024-09-21T07:26:23.218506+0000","last_fullsized":"2024-09-21T07:26:23.218506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:12.749885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217922+0000","last_change":"2024-09-21T07:26:15.750763+0000","last_active":"2024-09-21T07:26:23.217922+0000","last_peered":"2024-09-21T07:26:23.217922+0000","last_clean":"2024-09-21T07:26:23.217922+0000","last_became_active":"2024-09-21T07:26:15.750444+0000","last_became_peered":"2024-09-21T07:26:15.750444+0000","last_unstale":"2024-09-21T07:26:23.217922+0000","last_undegraded":"2024-09-21T07:26:23.217922+0000","last_fullsized":"2024-09-21T07:26:23.217922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792041+0000","last_change":"2024-09-21T07:26:14.745457+0000","last_active":"2024-09-21T07:26:21.792041+0000","last_peered":"2024-09-21T07:26:21.792041+0000","last_clean":"2024-09-21T07:26:21.792041+0000","last_became_active":"2024-09-21T07:26:14.745075+0000","last_became_peered":"2024-09-21T07:26:14.745075+0000","last_unstale":"2024-09-21T07:26:21.792041+0000","last_undegraded":"2024-09-21T07:26:21.792041+0000","last_fullsized":"2024-09-21T07:26:21.792041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:49.174372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756061+0000","last_change":"2024-09-21T07:26:20.202406+0000","last_active":"2024-09-21T07:26:22.756061+0000","last_peered":"2024-09-21T07:26:22.756061+0000","last_clean":"2024-09-21T07:26:22.756061+0000","last_became_active":"2024-09-21T07:26:20.201970+0000","last_became_peered":"2024-09-21T07:26:20.201970+0000","last_unstale":"2024-09-21T07:26:22.756061+0000","last_undegraded":"2024-09-21T07:26:22.756061+0000","last_fullsized":"2024-09-21T07:26:22.756061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901194+0000","last_change":"2024-09-21T07:26:15.769492+0000","last_active":"2024-09-21T07:26:22.901194+0000","last_peered":"2024-09-21T07:26:22.901194+0000","last_clean":"2024-09-21T07:26:22.901194+0000","last_became_active":"2024-09-21T07:26:15.768661+0000","last_became_peered":"2024-09-21T07:26:15.768661+0000","last_unstale":"2024-09-21T07:26:22.901194+0000","last_undegraded":"2024-09-21T07:26:22.901194+0000","last_fullsized":"2024-09-21T07:26:22.901194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:20.108036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218024+0000","last_change":"2024-09-21T07:26:14.737203+0000","last_active":"2024-09-21T07:26:23.218024+0000","last_peered":"2024-09-21T07:26:23.218024+0000","last_clean":"2024-09-21T07:26:23.218024+0000","last_became_active":"2024-09-21T07:26:14.736387+0000","last_became_peered":"2024-09-21T07:26:14.736387+0000","last_unstale":"2024-09-21T07:26:23.218024+0000","last_undegraded":"2024-09-21T07:26:23.218024+0000","last_fullsized":"2024-09-21T07:26:23.218024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217332+0000","last_change":"2024-09-21T07:26:19.793567+0000","last_active":"2024-09-21T07:26:23.217332+0000","last_peered":"2024-09-21T07:26:23.217332+0000","last_clean":"2024-09-21T07:26:23.217332+0000","last_became_active":"2024-09-21T07:26:19.793063+0000","last_became_peered":"2024-09-21T07:26:19.793063+0000","last_unstale":"2024-09-21T07:26:23.217332+0000","last_undegraded":"2024-09-21T07:26:23.217332+0000","last_fullsized":"2024-09-21T07:26:23.217332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:15.764293+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:15.763942+0000","last_became_peered":"2024-09-21T07:26:15.763942+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:40.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900887+0000","last_change":"2024-09-21T07:26:14.753757+0000","last_active":"2024-09-21T07:26:22.900887+0000","last_peered":"2024-09-21T07:26:22.900887+0000","last_clean":"2024-09-21T07:26:22.900887+0000","last_became_active":"2024-09-21T07:26:14.753415+0000","last_became_peered":"2024-09-21T07:26:14.753415+0000","last_unstale":"2024-09-21T07:26:22.900887+0000","last_undegraded":"2024-09-21T07:26:22.900887+0000","last_fullsized":"2024-09-21T07:26:22.900887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:48.397593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217762+0000","last_change":"2024-09-21T07:26:19.786373+0000","last_active":"2024-09-21T07:26:23.217762+0000","last_peered":"2024-09-21T07:26:23.217762+0000","last_clean":"2024-09-21T07:26:23.217762+0000","last_became_active":"2024-09-21T07:26:19.785794+0000","last_became_peered":"2024-09-21T07:26:19.785794+0000","last_unstale":"2024-09-21T07:26:23.217762+0000","last_undegraded":"2024-09-21T07:26:23.217762+0000","last_fullsized":"2024-09-21T07:26:23.217762+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216872+0000","last_change":"2024-09-21T07:26:15.752034+0000","last_active":"2024-09-21T07:26:23.216872+0000","last_peered":"2024-09-21T07:26:23.216872+0000","last_clean":"2024-09-21T07:26:23.216872+0000","last_became_active":"2024-09-21T07:26:15.751733+0000","last_became_peered":"2024-09-21T07:26:15.751733+0000","last_unstale":"2024-09-21T07:26:23.216872+0000","last_undegraded":"2024-09-21T07:26:23.216872+0000","last_fullsized":"2024-09-21T07:26:23.216872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789086+0000","last_change":"2024-09-21T07:26:14.741318+0000","last_active":"2024-09-21T07:26:21.789086+0000","last_peered":"2024-09-21T07:26:21.789086+0000","last_clean":"2024-09-21T07:26:21.789086+0000","last_became_active":"2024-09-21T07:26:14.740828+0000","last_became_peered":"2024-09-21T07:26:14.740828+0000","last_unstale":"2024-09-21T07:26:21.789086+0000","last_undegraded":"2024-09-21T07:26:21.789086+0000","last_fullsized":"2024-09-21T07:26:21.789086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.614543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218456+0000","last_change":"2024-09-21T07:26:19.793288+0000","last_active":"2024-09-21T07:26:23.218456+0000","last_peered":"2024-09-21T07:26:23.218456+0000","last_clean":"2024-09-21T07:26:23.218456+0000","last_became_active":"2024-09-21T07:26:19.792781+0000","last_became_peered":"2024-09-21T07:26:19.792781+0000","last_unstale":"2024-09-21T07:26:23.218456+0000","last_undegraded":"2024-09-21T07:26:23.218456+0000","last_fullsized":"2024-09-21T07:26:23.218456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755586+0000","last_change":"2024-09-21T07:26:15.763289+0000","last_active":"2024-09-21T07:26:22.755586+0000","last_peered":"2024-09-21T07:26:22.755586+0000","last_clean":"2024-09-21T07:26:22.755586+0000","last_became_active":"2024-09-21T07:26:15.762914+0000","last_became_peered":"2024-09-21T07:26:15.762914+0000","last_unstale":"2024-09-21T07:26:22.755586+0000","last_undegraded":"2024-09-21T07:26:22.755586+0000","last_fullsized":"2024-09-21T07:26:22.755586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:01.890170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790269+0000","last_change":"2024-09-21T07:26:14.746676+0000","last_active":"2024-09-21T07:26:21.790269+0000","last_peered":"2024-09-21T07:26:21.790269+0000","last_clean":"2024-09-21T07:26:21.790269+0000","last_became_active":"2024-09-21T07:26:14.745094+0000","last_became_peered":"2024-09-21T07:26:14.745094+0000","last_unstale":"2024-09-21T07:26:21.790269+0000","last_undegraded":"2024-09-21T07:26:21.790269+0000","last_fullsized":"2024-09-21T07:26:21.790269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:48.031389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756099+0000","last_change":"2024-09-21T07:26:20.202272+0000","last_active":"2024-09-21T07:26:22.756099+0000","last_peered":"2024-09-21T07:26:22.756099+0000","last_clean":"2024-09-21T07:26:22.756099+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:22.756099+0000","last_undegraded":"2024-09-21T07:26:22.756099+0000","last_fullsized":"2024-09-21T07:26:22.756099+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217063+0000","last_change":"2024-09-21T07:26:15.778494+0000","last_active":"2024-09-21T07:26:23.217063+0000","last_peered":"2024-09-21T07:26:23.217063+0000","last_clean":"2024-09-21T07:26:23.217063+0000","last_became_active":"2024-09-21T07:26:15.778117+0000","last_became_peered":"2024-09-21T07:26:15.778117+0000","last_unstale":"2024-09-21T07:26:23.217063+0000","last_undegraded":"2024-09-21T07:26:23.217063+0000","last_fullsized":"2024-09-21T07:26:23.217063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:52.614175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900190+0000","last_change":"2024-09-21T07:26:14.755827+0000","last_active":"2024-09-21T07:26:22.900190+0000","last_peered":"2024-09-21T07:26:22.900190+0000","last_clean":"2024-09-21T07:26:22.900190+0000","last_became_active":"2024-09-21T07:26:14.755319+0000","last_became_peered":"2024-09-21T07:26:14.755319+0000","last_unstale":"2024-09-21T07:26:22.900190+0000","last_undegraded":"2024-09-21T07:26:22.900190+0000","last_fullsized":"2024-09-21T07:26:22.900190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:31.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790381+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:26:21.790381+0000","last_peered":"2024-09-21T07:26:21.790381+0000","last_clean":"2024-09-21T07:26:21.790381+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:26:21.790381+0000","last_undegraded":"2024-09-21T07:26:21.790381+0000","last_fullsized":"2024-09-21T07:26:21.790381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790882+0000","last_change":"2024-09-21T07:26:20.205543+0000","last_active":"2024-09-21T07:26:21.790882+0000","last_peered":"2024-09-21T07:26:21.790882+0000","last_clean":"2024-09-21T07:26:21.790882+0000","last_became_active":"2024-09-21T07:26:20.204540+0000","last_became_peered":"2024-09-21T07:26:20.204540+0000","last_unstale":"2024-09-21T07:26:21.790882+0000","last_undegraded":"2024-09-21T07:26:21.790882+0000","last_fullsized":"2024-09-21T07:26:21.790882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791526+0000","last_change":"2024-09-21T07:26:15.776215+0000","last_active":"2024-09-21T07:26:21.791526+0000","last_peered":"2024-09-21T07:26:21.791526+0000","last_clean":"2024-09-21T07:26:21.791526+0000","last_became_active":"2024-09-21T07:26:15.775884+0000","last_became_peered":"2024-09-21T07:26:15.775884+0000","last_unstale":"2024-09-21T07:26:21.791526+0000","last_undegraded":"2024-09-21T07:26:21.791526+0000","last_fullsized":"2024-09-21T07:26:21.791526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:01.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789487+0000","last_change":"2024-09-21T07:26:14.756009+0000","last_active":"2024-09-21T07:26:21.789487+0000","last_peered":"2024-09-21T07:26:21.789487+0000","last_clean":"2024-09-21T07:26:21.789487+0000","last_became_active":"2024-09-21T07:26:14.755653+0000","last_became_peered":"2024-09-21T07:26:14.755653+0000","last_unstale":"2024-09-21T07:26:21.789487+0000","last_undegraded":"2024-09-21T07:26:21.789487+0000","last_fullsized":"2024-09-21T07:26:21.789487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:22.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790353+0000","last_change":"2024-09-21T07:26:20.202159+0000","last_active":"2024-09-21T07:26:21.790353+0000","last_peered":"2024-09-21T07:26:21.790353+0000","last_clean":"2024-09-21T07:26:21.790353+0000","last_became_active":"2024-09-21T07:26:20.201680+0000","last_became_peered":"2024-09-21T07:26:20.201680+0000","last_unstale":"2024-09-21T07:26:21.790353+0000","last_undegraded":"2024-09-21T07:26:21.790353+0000","last_fullsized":"2024-09-21T07:26:21.790353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:56.792767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901045+0000","last_change":"2024-09-21T07:26:15.771710+0000","last_active":"2024-09-21T07:26:22.901045+0000","last_peered":"2024-09-21T07:26:22.901045+0000","last_clean":"2024-09-21T07:26:22.901045+0000","last_became_active":"2024-09-21T07:26:15.770649+0000","last_became_peered":"2024-09-21T07:26:15.770649+0000","last_unstale":"2024-09-21T07:26:22.901045+0000","last_undegraded":"2024-09-21T07:26:22.901045+0000","last_fullsized":"2024-09-21T07:26:22.901045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.904168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218377+0000","last_change":"2024-09-21T07:26:14.740185+0000","last_active":"2024-09-21T07:26:23.218377+0000","last_peered":"2024-09-21T07:26:23.218377+0000","last_clean":"2024-09-21T07:26:23.218377+0000","last_became_active":"2024-09-21T07:26:14.738935+0000","last_became_peered":"2024-09-21T07:26:14.738935+0000","last_unstale":"2024-09-21T07:26:23.218377+0000","last_undegraded":"2024-09-21T07:26:23.218377+0000","last_fullsized":"2024-09-21T07:26:23.218377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756214+0000","last_change":"2024-09-21T07:26:19.789320+0000","last_active":"2024-09-21T07:26:22.756214+0000","last_peered":"2024-09-21T07:26:22.756214+0000","last_clean":"2024-09-21T07:26:22.756214+0000","last_became_active":"2024-09-21T07:26:19.788692+0000","last_became_peered":"2024-09-21T07:26:19.788692+0000","last_unstale":"2024-09-21T07:26:22.756214+0000","last_undegraded":"2024-09-21T07:26:22.756214+0000","last_fullsized":"2024-09-21T07:26:22.756214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792007+0000","last_change":"2024-09-21T07:26:15.775282+0000","last_active":"2024-09-21T07:26:21.792007+0000","last_peered":"2024-09-21T07:26:21.792007+0000","last_clean":"2024-09-21T07:26:21.792007+0000","last_became_active":"2024-09-21T07:26:15.775046+0000","last_became_peered":"2024-09-21T07:26:15.775046+0000","last_unstale":"2024-09-21T07:26:21.792007+0000","last_undegraded":"2024-09-21T07:26:21.792007+0000","last_fullsized":"2024-09-21T07:26:21.792007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:51.588092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789401+0000","last_change":"2024-09-21T07:26:14.740365+0000","last_active":"2024-09-21T07:26:21.789401+0000","last_peered":"2024-09-21T07:26:21.789401+0000","last_clean":"2024-09-21T07:26:21.789401+0000","last_became_active":"2024-09-21T07:26:14.739987+0000","last_became_peered":"2024-09-21T07:26:14.739987+0000","last_unstale":"2024-09-21T07:26:21.789401+0000","last_undegraded":"2024-09-21T07:26:21.789401+0000","last_fullsized":"2024-09-21T07:26:21.789401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900421+0000","last_change":"2024-09-21T07:26:20.205672+0000","last_active":"2024-09-21T07:26:22.900421+0000","last_peered":"2024-09-21T07:26:22.900421+0000","last_clean":"2024-09-21T07:26:22.900421+0000","last_became_active":"2024-09-21T07:26:20.205251+0000","last_became_peered":"2024-09-21T07:26:20.205251+0000","last_unstale":"2024-09-21T07:26:22.900421+0000","last_undegraded":"2024-09-21T07:26:22.900421+0000","last_fullsized":"2024-09-21T07:26:22.900421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:18.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790765+0000","last_change":"2024-09-21T07:26:15.762918+0000","last_active":"2024-09-21T07:26:21.790765+0000","last_peered":"2024-09-21T07:26:21.790765+0000","last_clean":"2024-09-21T07:26:21.790765+0000","last_became_active":"2024-09-21T07:26:15.762580+0000","last_became_peered":"2024-09-21T07:26:15.762580+0000","last_unstale":"2024-09-21T07:26:21.790765+0000","last_undegraded":"2024-09-21T07:26:21.790765+0000","last_fullsized":"2024-09-21T07:26:21.790765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.691687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218021+0000","last_change":"2024-09-21T07:26:14.739242+0000","last_active":"2024-09-21T07:26:23.218021+0000","last_peered":"2024-09-21T07:26:23.218021+0000","last_clean":"2024-09-21T07:26:23.218021+0000","last_became_active":"2024-09-21T07:26:14.737910+0000","last_became_peered":"2024-09-21T07:26:14.737910+0000","last_unstale":"2024-09-21T07:26:23.218021+0000","last_undegraded":"2024-09-21T07:26:23.218021+0000","last_fullsized":"2024-09-21T07:26:23.218021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:30.239449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217037+0000","last_change":"2024-09-21T07:26:19.782735+0000","last_active":"2024-09-21T07:26:23.217037+0000","last_peered":"2024-09-21T07:26:23.217037+0000","last_clean":"2024-09-21T07:26:23.217037+0000","last_became_active":"2024-09-21T07:26:19.782062+0000","last_became_peered":"2024-09-21T07:26:19.782062+0000","last_unstale":"2024-09-21T07:26:23.217037+0000","last_undegraded":"2024-09-21T07:26:23.217037+0000","last_fullsized":"2024-09-21T07:26:23.217037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790993+0000","last_change":"2024-09-21T07:26:15.772188+0000","last_active":"2024-09-21T07:26:21.790993+0000","last_peered":"2024-09-21T07:26:21.790993+0000","last_clean":"2024-09-21T07:26:21.790993+0000","last_became_active":"2024-09-21T07:26:15.771854+0000","last_became_peered":"2024-09-21T07:26:15.771854+0000","last_unstale":"2024-09-21T07:26:21.790993+0000","last_undegraded":"2024-09-21T07:26:21.790993+0000","last_fullsized":"2024-09-21T07:26:21.790993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:52.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.788951+0000","last_change":"2024-09-21T07:26:14.739783+0000","last_active":"2024-09-21T07:26:21.788951+0000","last_peered":"2024-09-21T07:26:21.788951+0000","last_clean":"2024-09-21T07:26:21.788951+0000","last_became_active":"2024-09-21T07:26:14.739433+0000","last_became_peered":"2024-09-21T07:26:14.739433+0000","last_unstale":"2024-09-21T07:26:21.788951+0000","last_undegraded":"2024-09-21T07:26:21.788951+0000","last_fullsized":"2024-09-21T07:26:21.788951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:08.827173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756045+0000","last_change":"2024-09-21T07:26:19.789353+0000","last_active":"2024-09-21T07:26:22.756045+0000","last_peered":"2024-09-21T07:26:22.756045+0000","last_clean":"2024-09-21T07:26:22.756045+0000","last_became_active":"2024-09-21T07:26:19.788890+0000","last_became_peered":"2024-09-21T07:26:19.788890+0000","last_unstale":"2024-09-21T07:26:22.756045+0000","last_undegraded":"2024-09-21T07:26:22.756045+0000","last_fullsized":"2024-09-21T07:26:22.756045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790254+0000","last_change":"2024-09-21T07:26:15.761846+0000","last_active":"2024-09-21T07:26:21.790254+0000","last_peered":"2024-09-21T07:26:21.790254+0000","last_clean":"2024-09-21T07:26:21.790254+0000","last_became_active":"2024-09-21T07:26:15.761125+0000","last_became_peered":"2024-09-21T07:26:15.761125+0000","last_unstale":"2024-09-21T07:26:21.790254+0000","last_undegraded":"2024-09-21T07:26:21.790254+0000","last_fullsized":"2024-09-21T07:26:21.790254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791303+0000","last_change":"2024-09-21T07:26:14.751611+0000","last_active":"2024-09-21T07:26:21.791303+0000","last_peered":"2024-09-21T07:26:21.791303+0000","last_clean":"2024-09-21T07:26:21.791303+0000","last_became_active":"2024-09-21T07:26:14.750246+0000","last_became_peered":"2024-09-21T07:26:14.750246+0000","last_unstale":"2024-09-21T07:26:21.791303+0000","last_undegraded":"2024-09-21T07:26:21.791303+0000","last_fullsized":"2024-09-21T07:26:21.791303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:45.761871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218074+0000","last_change":"2024-09-21T07:26:20.202567+0000","last_active":"2024-09-21T07:26:23.218074+0000","last_peered":"2024-09-21T07:26:23.218074+0000","last_clean":"2024-09-21T07:26:23.218074+0000","last_became_active":"2024-09-21T07:26:20.201637+0000","last_became_peered":"2024-09-21T07:26:20.201637+0000","last_unstale":"2024-09-21T07:26:23.218074+0000","last_undegraded":"2024-09-21T07:26:23.218074+0000","last_fullsized":"2024-09-21T07:26:23.218074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:17.143053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791707+0000","last_change":"2024-09-21T07:26:15.775390+0000","last_active":"2024-09-21T07:26:21.791707+0000","last_peered":"2024-09-21T07:26:21.791707+0000","last_clean":"2024-09-21T07:26:21.791707+0000","last_became_active":"2024-09-21T07:26:15.774972+0000","last_became_peered":"2024-09-21T07:26:15.774972+0000","last_unstale":"2024-09-21T07:26:21.791707+0000","last_undegraded":"2024-09-21T07:26:21.791707+0000","last_fullsized":"2024-09-21T07:26:21.791707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.551786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900819+0000","last_change":"2024-09-21T07:26:14.752809+0000","last_active":"2024-09-21T07:26:22.900819+0000","last_peered":"2024-09-21T07:26:22.900819+0000","last_clean":"2024-09-21T07:26:22.900819+0000","last_became_active":"2024-09-21T07:26:14.752442+0000","last_became_peered":"2024-09-21T07:26:14.752442+0000","last_unstale":"2024-09-21T07:26:22.900819+0000","last_undegraded":"2024-09-21T07:26:22.900819+0000","last_fullsized":"2024-09-21T07:26:22.900819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.370404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218055+0000","last_change":"2024-09-21T07:26:20.202097+0000","last_active":"2024-09-21T07:26:23.218055+0000","last_peered":"2024-09-21T07:26:23.218055+0000","last_clean":"2024-09-21T07:26:23.218055+0000","last_became_active":"2024-09-21T07:26:20.201641+0000","last_became_peered":"2024-09-21T07:26:20.201641+0000","last_unstale":"2024-09-21T07:26:23.218055+0000","last_undegraded":"2024-09-21T07:26:23.218055+0000","last_fullsized":"2024-09-21T07:26:23.218055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:52.838818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755771+0000","last_change":"2024-09-21T07:26:15.763851+0000","last_active":"2024-09-21T07:26:22.755771+0000","last_peered":"2024-09-21T07:26:22.755771+0000","last_clean":"2024-09-21T07:26:22.755771+0000","last_became_active":"2024-09-21T07:26:15.763088+0000","last_became_peered":"2024-09-21T07:26:15.763088+0000","last_unstale":"2024-09-21T07:26:22.755771+0000","last_undegraded":"2024-09-21T07:26:22.755771+0000","last_fullsized":"2024-09-21T07:26:22.755771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:00.411908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756372+0000","last_change":"2024-09-21T07:26:14.749147+0000","last_active":"2024-09-21T07:26:22.756372+0000","last_peered":"2024-09-21T07:26:22.756372+0000","last_clean":"2024-09-21T07:26:22.756372+0000","last_became_active":"2024-09-21T07:26:14.748858+0000","last_became_peered":"2024-09-21T07:26:14.748858+0000","last_unstale":"2024-09-21T07:26:22.756372+0000","last_undegraded":"2024-09-21T07:26:22.756372+0000","last_fullsized":"2024-09-21T07:26:22.756372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:19.994800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901148+0000","last_change":"2024-09-21T07:26:20.202180+0000","last_active":"2024-09-21T07:26:22.901148+0000","last_peered":"2024-09-21T07:26:22.901148+0000","last_clean":"2024-09-21T07:26:22.901148+0000","last_became_active":"2024-09-21T07:26:20.201665+0000","last_became_peered":"2024-09-21T07:26:20.201665+0000","last_unstale":"2024-09-21T07:26:22.901148+0000","last_undegraded":"2024-09-21T07:26:22.901148+0000","last_fullsized":"2024-09-21T07:26:22.901148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:49.726125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.763407+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.763087+0000","last_became_peered":"2024-09-21T07:26:15.763087+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:49.061741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:26.613007+0000","last_change":"2024-09-21T07:26:14.746399+0000","last_active":"2024-09-21T07:26:26.613007+0000","last_peered":"2024-09-21T07:26:26.613007+0000","last_clean":"2024-09-21T07:26:26.613007+0000","last_became_active":"2024-09-21T07:26:14.746029+0000","last_became_peered":"2024-09-21T07:26:14.746029+0000","last_unstale":"2024-09-21T07:26:26.613007+0000","last_undegraded":"2024-09-21T07:26:26.613007+0000","last_fullsized":"2024-09-21T07:26:26.613007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:47.548805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217015+0000","last_change":"2024-09-21T07:26:19.793971+0000","last_active":"2024-09-21T07:26:23.217015+0000","last_peered":"2024-09-21T07:26:23.217015+0000","last_clean":"2024-09-21T07:26:23.217015+0000","last_became_active":"2024-09-21T07:26:19.793342+0000","last_became_peered":"2024-09-21T07:26:19.793342+0000","last_unstale":"2024-09-21T07:26:23.217015+0000","last_undegraded":"2024-09-21T07:26:23.217015+0000","last_fullsized":"2024-09-21T07:26:23.217015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217997+0000","last_change":"2024-09-21T07:26:15.763698+0000","last_active":"2024-09-21T07:26:23.217997+0000","last_peered":"2024-09-21T07:26:23.217997+0000","last_clean":"2024-09-21T07:26:23.217997+0000","last_became_active":"2024-09-21T07:26:15.763010+0000","last_became_peered":"2024-09-21T07:26:15.763010+0000","last_unstale":"2024-09-21T07:26:23.217997+0000","last_undegraded":"2024-09-21T07:26:23.217997+0000","last_fullsized":"2024-09-21T07:26:23.217997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:49.038621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218340+0000","last_change":"2024-09-21T07:26:14.756162+0000","last_active":"2024-09-21T07:26:23.218340+0000","last_peered":"2024-09-21T07:26:23.218340+0000","last_clean":"2024-09-21T07:26:23.218340+0000","last_became_active":"2024-09-21T07:26:14.755821+0000","last_became_peered":"2024-09-21T07:26:14.755821+0000","last_unstale":"2024-09-21T07:26:23.218340+0000","last_undegraded":"2024-09-21T07:26:23.218340+0000","last_fullsized":"2024-09-21T07:26:23.218340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217766+0000","last_change":"2024-09-21T07:26:19.782715+0000","last_active":"2024-09-21T07:26:23.217766+0000","last_peered":"2024-09-21T07:26:23.217766+0000","last_clean":"2024-09-21T07:26:23.217766+0000","last_became_active":"2024-09-21T07:26:19.782285+0000","last_became_peered":"2024-09-21T07:26:19.782285+0000","last_unstale":"2024-09-21T07:26:23.217766+0000","last_undegraded":"2024-09-21T07:26:23.217766+0000","last_fullsized":"2024-09-21T07:26:23.217766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756175+0000","last_change":"2024-09-21T07:26:15.759805+0000","last_active":"2024-09-21T07:26:22.756175+0000","last_peered":"2024-09-21T07:26:22.756175+0000","last_clean":"2024-09-21T07:26:22.756175+0000","last_became_active":"2024-09-21T07:26:15.759487+0000","last_became_peered":"2024-09-21T07:26:15.759487+0000","last_unstale":"2024-09-21T07:26:22.756175+0000","last_undegraded":"2024-09-21T07:26:22.756175+0000","last_fullsized":"2024-09-21T07:26:22.756175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900206+0000","last_change":"2024-09-21T07:26:14.753655+0000","last_active":"2024-09-21T07:26:22.900206+0000","last_peered":"2024-09-21T07:26:22.900206+0000","last_clean":"2024-09-21T07:26:22.900206+0000","last_became_active":"2024-09-21T07:26:14.753266+0000","last_became_peered":"2024-09-21T07:26:14.753266+0000","last_unstale":"2024-09-21T07:26:22.900206+0000","last_undegraded":"2024-09-21T07:26:22.900206+0000","last_fullsized":"2024-09-21T07:26:22.900206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:14.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218754+0000","last_change":"2024-09-21T07:26:19.786292+0000","last_active":"2024-09-21T07:26:23.218754+0000","last_peered":"2024-09-21T07:26:23.218754+0000","last_clean":"2024-09-21T07:26:23.218754+0000","last_became_active":"2024-09-21T07:26:19.783800+0000","last_became_peered":"2024-09-21T07:26:19.783800+0000","last_unstale":"2024-09-21T07:26:23.218754+0000","last_undegraded":"2024-09-21T07:26:23.218754+0000","last_fullsized":"2024-09-21T07:26:23.218754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:52.992488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901112+0000","last_change":"2024-09-21T07:26:15.766957+0000","last_active":"2024-09-21T07:26:22.901112+0000","last_peered":"2024-09-21T07:26:22.901112+0000","last_clean":"2024-09-21T07:26:22.901112+0000","last_became_active":"2024-09-21T07:26:15.766640+0000","last_became_peered":"2024-09-21T07:26:15.766640+0000","last_unstale":"2024-09-21T07:26:22.901112+0000","last_undegraded":"2024-09-21T07:26:22.901112+0000","last_fullsized":"2024-09-21T07:26:22.901112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:23.200622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791558+0000","last_change":"2024-09-21T07:26:14.756852+0000","last_active":"2024-09-21T07:26:21.791558+0000","last_peered":"2024-09-21T07:26:21.791558+0000","last_clean":"2024-09-21T07:26:21.791558+0000","last_became_active":"2024-09-21T07:26:14.756512+0000","last_became_peered":"2024-09-21T07:26:14.756512+0000","last_unstale":"2024-09-21T07:26:21.791558+0000","last_undegraded":"2024-09-21T07:26:21.791558+0000","last_fullsized":"2024-09-21T07:26:21.791558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217003+0000","last_change":"2024-09-21T07:26:20.205385+0000","last_active":"2024-09-21T07:26:23.217003+0000","last_peered":"2024-09-21T07:26:23.217003+0000","last_clean":"2024-09-21T07:26:23.217003+0000","last_became_active":"2024-09-21T07:26:20.205016+0000","last_became_peered":"2024-09-21T07:26:20.205016+0000","last_unstale":"2024-09-21T07:26:23.217003+0000","last_undegraded":"2024-09-21T07:26:23.217003+0000","last_fullsized":"2024-09-21T07:26:23.217003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.612156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755846+0000","last_change":"2024-09-21T07:26:20.202075+0000","last_active":"2024-09-21T07:26:22.755846+0000","last_peered":"2024-09-21T07:26:22.755846+0000","last_clean":"2024-09-21T07:26:22.755846+0000","last_became_active":"2024-09-21T07:26:20.201632+0000","last_became_peered":"2024-09-21T07:26:20.201632+0000","last_unstale":"2024-09-21T07:26:22.755846+0000","last_undegraded":"2024-09-21T07:26:22.755846+0000","last_fullsized":"2024-09-21T07:26:22.755846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216493+0000","last_change":"2024-09-21T07:26:15.762928+0000","last_active":"2024-09-21T07:26:23.216493+0000","last_peered":"2024-09-21T07:26:23.216493+0000","last_clean":"2024-09-21T07:26:23.216493+0000","last_became_active":"2024-09-21T07:26:15.762291+0000","last_became_peered":"2024-09-21T07:26:15.762291+0000","last_unstale":"2024-09-21T07:26:23.216493+0000","last_undegraded":"2024-09-21T07:26:23.216493+0000","last_fullsized":"2024-09-21T07:26:23.216493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900637+0000","last_change":"2024-09-21T07:26:14.755764+0000","last_active":"2024-09-21T07:26:22.900637+0000","last_peered":"2024-09-21T07:26:22.900637+0000","last_clean":"2024-09-21T07:26:22.900637+0000","last_became_active":"2024-09-21T07:26:14.755397+0000","last_became_peered":"2024-09-21T07:26:14.755397+0000","last_unstale":"2024-09-21T07:26:22.900637+0000","last_undegraded":"2024-09-21T07:26:22.900637+0000","last_fullsized":"2024-09-21T07:26:22.900637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:32.946211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791301+0000","last_change":"2024-09-21T07:26:19.789958+0000","last_active":"2024-09-21T07:26:21.791301+0000","last_peered":"2024-09-21T07:26:21.791301+0000","last_clean":"2024-09-21T07:26:21.791301+0000","last_became_active":"2024-09-21T07:26:19.789515+0000","last_became_peered":"2024-09-21T07:26:19.789515+0000","last_unstale":"2024-09-21T07:26:21.791301+0000","last_undegraded":"2024-09-21T07:26:21.791301+0000","last_fullsized":"2024-09-21T07:26:21.791301+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218685+0000","last_change":"2024-09-21T07:26:15.760941+0000","last_active":"2024-09-21T07:26:23.218685+0000","last_peered":"2024-09-21T07:26:23.218685+0000","last_clean":"2024-09-21T07:26:23.218685+0000","last_became_active":"2024-09-21T07:26:15.760612+0000","last_became_peered":"2024-09-21T07:26:15.760612+0000","last_unstale":"2024-09-21T07:26:23.218685+0000","last_undegraded":"2024-09-21T07:26:23.218685+0000","last_fullsized":"2024-09-21T07:26:23.218685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:23.610491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216574+0000","last_change":"2024-09-21T07:26:14.744602+0000","last_active":"2024-09-21T07:26:23.216574+0000","last_peered":"2024-09-21T07:26:23.216574+0000","last_clean":"2024-09-21T07:26:23.216574+0000","last_became_active":"2024-09-21T07:26:14.744284+0000","last_became_peered":"2024-09-21T07:26:14.744284+0000","last_unstale":"2024-09-21T07:26:23.216574+0000","last_undegraded":"2024-09-21T07:26:23.216574+0000","last_fullsized":"2024-09-21T07:26:23.216574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:39.642931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790820+0000","last_change":"2024-09-21T07:26:14.750830+0000","last_active":"2024-09-21T07:26:21.790820+0000","last_peered":"2024-09-21T07:26:21.790820+0000","last_clean":"2024-09-21T07:26:21.790820+0000","last_became_active":"2024-09-21T07:26:14.749815+0000","last_became_peered":"2024-09-21T07:26:14.749815+0000","last_unstale":"2024-09-21T07:26:21.790820+0000","last_undegraded":"2024-09-21T07:26:21.790820+0000","last_fullsized":"2024-09-21T07:26:21.790820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756340+0000","last_change":"2024-09-21T07:26:15.773671+0000","last_active":"2024-09-21T07:26:22.756340+0000","last_peered":"2024-09-21T07:26:22.756340+0000","last_clean":"2024-09-21T07:26:22.756340+0000","last_became_active":"2024-09-21T07:26:15.773380+0000","last_became_peered":"2024-09-21T07:26:15.773380+0000","last_unstale":"2024-09-21T07:26:22.756340+0000","last_undegraded":"2024-09-21T07:26:22.756340+0000","last_fullsized":"2024-09-21T07:26:22.756340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789217+0000","last_change":"2024-09-21T07:26:19.785278+0000","last_active":"2024-09-21T07:26:21.789217+0000","last_peered":"2024-09-21T07:26:21.789217+0000","last_clean":"2024-09-21T07:26:21.789217+0000","last_became_active":"2024-09-21T07:26:19.784142+0000","last_became_peered":"2024-09-21T07:26:19.784142+0000","last_unstale":"2024-09-21T07:26:21.789217+0000","last_undegraded":"2024-09-21T07:26:21.789217+0000","last_fullsized":"2024-09-21T07:26:21.789217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.144434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217614+0000","last_change":"2024-09-21T07:26:14.746012+0000","last_active":"2024-09-21T07:26:23.217614+0000","last_peered":"2024-09-21T07:26:23.217614+0000","last_clean":"2024-09-21T07:26:23.217614+0000","last_became_active":"2024-09-21T07:26:14.745668+0000","last_became_peered":"2024-09-21T07:26:14.745668+0000","last_unstale":"2024-09-21T07:26:23.217614+0000","last_undegraded":"2024-09-21T07:26:23.217614+0000","last_fullsized":"2024-09-21T07:26:23.217614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:03.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217510+0000","last_change":"2024-09-21T07:26:15.764689+0000","last_active":"2024-09-21T07:26:23.217510+0000","last_peered":"2024-09-21T07:26:23.217510+0000","last_clean":"2024-09-21T07:26:23.217510+0000","last_became_active":"2024-09-21T07:26:15.763516+0000","last_became_peered":"2024-09-21T07:26:15.763516+0000","last_unstale":"2024-09-21T07:26:23.217510+0000","last_undegraded":"2024-09-21T07:26:23.217510+0000","last_fullsized":"2024-09-21T07:26:23.217510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.556036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217467+0000","last_change":"2024-09-21T07:26:19.784787+0000","last_active":"2024-09-21T07:26:23.217467+0000","last_peered":"2024-09-21T07:26:23.217467+0000","last_clean":"2024-09-21T07:26:23.217467+0000","last_became_active":"2024-09-21T07:26:19.784262+0000","last_became_peered":"2024-09-21T07:26:19.784262+0000","last_unstale":"2024-09-21T07:26:23.217467+0000","last_undegraded":"2024-09-21T07:26:23.217467+0000","last_fullsized":"2024-09-21T07:26:23.217467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:47.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790131+0000","last_change":"2024-09-21T07:26:15.762178+0000","last_active":"2024-09-21T07:26:21.790131+0000","last_peered":"2024-09-21T07:26:21.790131+0000","last_clean":"2024-09-21T07:26:21.790131+0000","last_became_active":"2024-09-21T07:26:15.761535+0000","last_became_peered":"2024-09-21T07:26:15.761535+0000","last_unstale":"2024-09-21T07:26:21.790131+0000","last_undegraded":"2024-09-21T07:26:21.790131+0000","last_fullsized":"2024-09-21T07:26:21.790131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789216+0000","last_change":"2024-09-21T07:26:15.760862+0000","last_active":"2024-09-21T07:26:21.789216+0000","last_peered":"2024-09-21T07:26:21.789216+0000","last_clean":"2024-09-21T07:26:21.789216+0000","last_became_active":"2024-09-21T07:26:15.760547+0000","last_became_peered":"2024-09-21T07:26:15.760547+0000","last_unstale":"2024-09-21T07:26:21.789216+0000","last_undegraded":"2024-09-21T07:26:21.789216+0000","last_fullsized":"2024-09-21T07:26:21.789216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:49.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900702+0000","last_change":"2024-09-21T07:26:15.773293+0000","last_active":"2024-09-21T07:26:22.900702+0000","last_peered":"2024-09-21T07:26:22.900702+0000","last_clean":"2024-09-21T07:26:22.900702+0000","last_became_active":"2024-09-21T07:26:15.772904+0000","last_became_peered":"2024-09-21T07:26:15.772904+0000","last_unstale":"2024-09-21T07:26:22.900702+0000","last_undegraded":"2024-09-21T07:26:22.900702+0000","last_fullsized":"2024-09-21T07:26:22.900702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:56.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756234+0000","last_change":"2024-09-21T07:26:15.764081+0000","last_active":"2024-09-21T07:26:22.756234+0000","last_peered":"2024-09-21T07:26:22.756234+0000","last_clean":"2024-09-21T07:26:22.756234+0000","last_became_active":"2024-09-21T07:26:15.763052+0000","last_became_peered":"2024-09-21T07:26:15.763052+0000","last_unstale":"2024-09-21T07:26:22.756234+0000","last_undegraded":"2024-09-21T07:26:22.756234+0000","last_fullsized":"2024-09-21T07:26:22.756234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218342+0000","last_change":"2024-09-21T07:26:15.766994+0000","last_active":"2024-09-21T07:26:23.218342+0000","last_peered":"2024-09-21T07:26:23.218342+0000","last_clean":"2024-09-21T07:26:23.218342+0000","last_became_active":"2024-09-21T07:26:15.766623+0000","last_became_peered":"2024-09-21T07:26:15.766623+0000","last_unstale":"2024-09-21T07:26:23.218342+0000","last_undegraded":"2024-09-21T07:26:23.218342+0000","last_fullsized":"2024-09-21T07:26:23.218342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:59.716664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218240+0000","last_change":"2024-09-21T07:26:15.758623+0000","last_active":"2024-09-21T07:26:23.218240+0000","last_peered":"2024-09-21T07:26:23.218240+0000","last_clean":"2024-09-21T07:26:23.218240+0000","last_became_active":"2024-09-21T07:26:15.758309+0000","last_became_peered":"2024-09-21T07:26:15.758309+0000","last_unstale":"2024-09-21T07:26:23.218240+0000","last_undegraded":"2024-09-21T07:26:23.218240+0000","last_fullsized":"2024-09-21T07:26:23.218240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792006+0000","last_change":"2024-09-21T07:26:15.770009+0000","last_active":"2024-09-21T07:26:21.792006+0000","last_peered":"2024-09-21T07:26:21.792006+0000","last_clean":"2024-09-21T07:26:21.792006+0000","last_became_active":"2024-09-21T07:26:15.769549+0000","last_became_peered":"2024-09-21T07:26:15.769549+0000","last_unstale":"2024-09-21T07:26:21.792006+0000","last_undegraded":"2024-09-21T07:26:21.792006+0000","last_fullsized":"2024-09-21T07:26:21.792006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218223+0000","last_change":"2024-09-21T07:26:15.765956+0000","last_active":"2024-09-21T07:26:23.218223+0000","last_peered":"2024-09-21T07:26:23.218223+0000","last_clean":"2024-09-21T07:26:23.218223+0000","last_became_active":"2024-09-21T07:26:15.765561+0000","last_became_peered":"2024-09-21T07:26:15.765561+0000","last_unstale":"2024-09-21T07:26:23.218223+0000","last_undegraded":"2024-09-21T07:26:23.218223+0000","last_fullsized":"2024-09-21T07:26:23.218223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:35.944784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900539+0000","last_change":"2024-09-21T07:26:15.766091+0000","last_active":"2024-09-21T07:26:22.900539+0000","last_peered":"2024-09-21T07:26:22.900539+0000","last_clean":"2024-09-21T07:26:22.900539+0000","last_became_active":"2024-09-21T07:26:15.765771+0000","last_became_peered":"2024-09-21T07:26:15.765771+0000","last_unstale":"2024-09-21T07:26:22.900539+0000","last_undegraded":"2024-09-21T07:26:22.900539+0000","last_fullsized":"2024-09-21T07:26:22.900539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:15.463622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217240+0000","last_change":"2024-09-21T07:26:15.771905+0000","last_active":"2024-09-21T07:26:23.217240+0000","last_peered":"2024-09-21T07:26:23.217240+0000","last_clean":"2024-09-21T07:26:23.217240+0000","last_became_active":"2024-09-21T07:26:15.771525+0000","last_became_peered":"2024-09-21T07:26:15.771525+0000","last_unstale":"2024-09-21T07:26:23.217240+0000","last_undegraded":"2024-09-21T07:26:23.217240+0000","last_fullsized":"2024-09-21T07:26:23.217240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.988043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900725+0000","last_change":"2024-09-21T07:26:15.775865+0000","last_active":"2024-09-21T07:26:22.900725+0000","last_peered":"2024-09-21T07:26:22.900725+0000","last_clean":"2024-09-21T07:26:22.900725+0000","last_became_active":"2024-09-21T07:26:15.773173+0000","last_became_peered":"2024-09-21T07:26:15.773173+0000","last_unstale":"2024-09-21T07:26:22.900725+0000","last_undegraded":"2024-09-21T07:26:22.900725+0000","last_fullsized":"2024-09-21T07:26:22.900725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:42.898368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756041+0000","last_change":"2024-09-21T07:26:15.766057+0000","last_active":"2024-09-21T07:26:22.756041+0000","last_peered":"2024-09-21T07:26:22.756041+0000","last_clean":"2024-09-21T07:26:22.756041+0000","last_became_active":"2024-09-21T07:26:15.765714+0000","last_became_peered":"2024-09-21T07:26:15.765714+0000","last_unstale":"2024-09-21T07:26:22.756041+0000","last_undegraded":"2024-09-21T07:26:22.756041+0000","last_fullsized":"2024-09-21T07:26:22.756041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790549+0000","last_change":"2024-09-21T07:26:15.774046+0000","last_active":"2024-09-21T07:26:21.790549+0000","last_peered":"2024-09-21T07:26:21.790549+0000","last_clean":"2024-09-21T07:26:21.790549+0000","last_became_active":"2024-09-21T07:26:15.773709+0000","last_became_peered":"2024-09-21T07:26:15.773709+0000","last_unstale":"2024-09-21T07:26:21.790549+0000","last_undegraded":"2024-09-21T07:26:21.790549+0000","last_fullsized":"2024-09-21T07:26:21.790549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791898+0000","last_change":"2024-09-21T07:26:15.760349+0000","last_active":"2024-09-21T07:26:21.791898+0000","last_peered":"2024-09-21T07:26:21.791898+0000","last_clean":"2024-09-21T07:26:21.791898+0000","last_became_active":"2024-09-21T07:26:15.759759+0000","last_became_peered":"2024-09-21T07:26:15.759759+0000","last_unstale":"2024-09-21T07:26:21.791898+0000","last_undegraded":"2024-09-21T07:26:21.791898+0000","last_fullsized":"2024-09-21T07:26:21.791898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:33.433481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789376+0000","last_change":"2024-09-21T07:26:15.767539+0000","last_active":"2024-09-21T07:26:21.789376+0000","last_peered":"2024-09-21T07:26:21.789376+0000","last_clean":"2024-09-21T07:26:21.789376+0000","last_became_active":"2024-09-21T07:26:15.767174+0000","last_became_peered":"2024-09-21T07:26:15.767174+0000","last_unstale":"2024-09-21T07:26:21.789376+0000","last_undegraded":"2024-09-21T07:26:21.789376+0000","last_fullsized":"2024-09-21T07:26:21.789376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:08.568390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901369+0000","last_change":"2024-09-21T07:26:15.767788+0000","last_active":"2024-09-21T07:26:22.901369+0000","last_peered":"2024-09-21T07:26:22.901369+0000","last_clean":"2024-09-21T07:26:22.901369+0000","last_became_active":"2024-09-21T07:26:15.767459+0000","last_became_peered":"2024-09-21T07:26:15.767459+0000","last_unstale":"2024-09-21T07:26:22.901369+0000","last_undegraded":"2024-09-21T07:26:22.901369+0000","last_fullsized":"2024-09-21T07:26:22.901369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.318947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900393+0000","last_change":"2024-09-21T07:26:15.758591+0000","last_active":"2024-09-21T07:26:22.900393+0000","last_peered":"2024-09-21T07:26:22.900393+0000","last_clean":"2024-09-21T07:26:22.900393+0000","last_became_active":"2024-09-21T07:26:15.758248+0000","last_became_peered":"2024-09-21T07:26:15.758248+0000","last_unstale":"2024-09-21T07:26:22.900393+0000","last_undegraded":"2024-09-21T07:26:22.900393+0000","last_fullsized":"2024-09-21T07:26:22.900393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:36.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216601+0000","last_change":"2024-09-21T07:26:15.763665+0000","last_active":"2024-09-21T07:26:23.216601+0000","last_peered":"2024-09-21T07:26:23.216601+0000","last_clean":"2024-09-21T07:26:23.216601+0000","last_became_active":"2024-09-21T07:26:15.763348+0000","last_became_peered":"2024-09-21T07:26:15.763348+0000","last_unstale":"2024-09-21T07:26:23.216601+0000","last_undegraded":"2024-09-21T07:26:23.216601+0000","last_fullsized":"2024-09-21T07:26:23.216601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755781+0000","last_change":"2024-09-21T07:26:15.757427+0000","last_active":"2024-09-21T07:26:22.755781+0000","last_peered":"2024-09-21T07:26:22.755781+0000","last_clean":"2024-09-21T07:26:22.755781+0000","last_became_active":"2024-09-21T07:26:15.757078+0000","last_became_peered":"2024-09-21T07:26:15.757078+0000","last_unstale":"2024-09-21T07:26:22.755781+0000","last_undegraded":"2024-09-21T07:26:22.755781+0000","last_fullsized":"2024-09-21T07:26:22.755781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:46.647829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:15.767064+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:15.766690+0000","last_became_peered":"2024-09-21T07:26:15.766690+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:07.860716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217825+0000","last_change":"2024-09-21T07:26:15.775975+0000","last_active":"2024-09-21T07:26:23.217825+0000","last_peered":"2024-09-21T07:26:23.217825+0000","last_clean":"2024-09-21T07:26:23.217825+0000","last_became_active":"2024-09-21T07:26:15.775526+0000","last_became_peered":"2024-09-21T07:26:15.775526+0000","last_unstale":"2024-09-21T07:26:23.217825+0000","last_undegraded":"2024-09-21T07:26:23.217825+0000","last_fullsized":"2024-09-21T07:26:23.217825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.840733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791070+0000","last_change":"2024-09-21T07:26:15.776942+0000","last_active":"2024-09-21T07:26:21.791070+0000","last_peered":"2024-09-21T07:26:21.791070+0000","last_clean":"2024-09-21T07:26:21.791070+0000","last_became_active":"2024-09-21T07:26:15.776632+0000","last_became_peered":"2024-09-21T07:26:15.776632+0000","last_unstale":"2024-09-21T07:26:21.791070+0000","last_undegraded":"2024-09-21T07:26:21.791070+0000","last_fullsized":"2024-09-21T07:26:21.791070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791878+0000","last_change":"2024-09-21T07:26:15.766327+0000","last_active":"2024-09-21T07:26:21.791878+0000","last_peered":"2024-09-21T07:26:21.791878+0000","last_clean":"2024-09-21T07:26:21.791878+0000","last_became_active":"2024-09-21T07:26:15.765981+0000","last_became_peered":"2024-09-21T07:26:15.765981+0000","last_unstale":"2024-09-21T07:26:21.791878+0000","last_undegraded":"2024-09-21T07:26:21.791878+0000","last_fullsized":"2024-09-21T07:26:21.791878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791919+0000","last_change":"2024-09-21T07:26:15.777412+0000","last_active":"2024-09-21T07:26:21.791919+0000","last_peered":"2024-09-21T07:26:21.791919+0000","last_clean":"2024-09-21T07:26:21.791919+0000","last_became_active":"2024-09-21T07:26:15.776925+0000","last_became_peered":"2024-09-21T07:26:15.776925+0000","last_unstale":"2024-09-21T07:26:21.791919+0000","last_undegraded":"2024-09-21T07:26:21.791919+0000","last_fullsized":"2024-09-21T07:26:21.791919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:38.270679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218556+0000","last_change":"2024-09-21T07:26:15.777075+0000","last_active":"2024-09-21T07:26:23.218556+0000","last_peered":"2024-09-21T07:26:23.218556+0000","last_clean":"2024-09-21T07:26:23.218556+0000","last_became_active":"2024-09-21T07:26:15.776611+0000","last_became_peered":"2024-09-21T07:26:15.776611+0000","last_unstale":"2024-09-21T07:26:23.218556+0000","last_undegraded":"2024-09-21T07:26:23.218556+0000","last_fullsized":"2024-09-21T07:26:23.218556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:38.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790866+0000","last_change":"2024-09-21T07:26:15.762408+0000","last_active":"2024-09-21T07:26:21.790866+0000","last_peered":"2024-09-21T07:26:21.790866+0000","last_clean":"2024-09-21T07:26:21.790866+0000","last_became_active":"2024-09-21T07:26:15.762014+0000","last_became_peered":"2024-09-21T07:26:15.762014+0000","last_unstale":"2024-09-21T07:26:21.790866+0000","last_undegraded":"2024-09-21T07:26:21.790866+0000","last_fullsized":"2024-09-21T07:26:21.790866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:09.186226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900571+0000","last_change":"2024-09-21T07:26:15.759759+0000","last_active":"2024-09-21T07:26:22.900571+0000","last_peered":"2024-09-21T07:26:22.900571+0000","last_clean":"2024-09-21T07:26:22.900571+0000","last_became_active":"2024-09-21T07:26:15.758871+0000","last_became_peered":"2024-09-21T07:26:15.758871+0000","last_unstale":"2024-09-21T07:26:22.900571+0000","last_undegraded":"2024-09-21T07:26:22.900571+0000","last_fullsized":"2024-09-21T07:26:22.900571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:23.666201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791461+0000","last_change":"2024-09-21T07:26:15.768245+0000","last_active":"2024-09-21T07:26:21.791461+0000","last_peered":"2024-09-21T07:26:21.791461+0000","last_clean":"2024-09-21T07:26:21.791461+0000","last_became_active":"2024-09-21T07:26:15.767901+0000","last_became_peered":"2024-09-21T07:26:15.767901+0000","last_unstale":"2024-09-21T07:26:21.791461+0000","last_undegraded":"2024-09-21T07:26:21.791461+0000","last_fullsized":"2024-09-21T07:26:21.791461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:52.823399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900922+0000","last_change":"2024-09-21T07:26:15.772040+0000","last_active":"2024-09-21T07:26:22.900922+0000","last_peered":"2024-09-21T07:26:22.900922+0000","last_clean":"2024-09-21T07:26:22.900922+0000","last_became_active":"2024-09-21T07:26:15.771194+0000","last_became_peered":"2024-09-21T07:26:15.771194+0000","last_unstale":"2024-09-21T07:26:22.900922+0000","last_undegraded":"2024-09-21T07:26:22.900922+0000","last_fullsized":"2024-09-21T07:26:22.900922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:35.969051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755834+0000","last_change":"2024-09-21T07:26:15.752787+0000","last_active":"2024-09-21T07:26:22.755834+0000","last_peered":"2024-09-21T07:26:22.755834+0000","last_clean":"2024-09-21T07:26:22.755834+0000","last_became_active":"2024-09-21T07:26:15.752457+0000","last_became_peered":"2024-09-21T07:26:15.752457+0000","last_unstale":"2024-09-21T07:26:22.755834+0000","last_undegraded":"2024-09-21T07:26:22.755834+0000","last_fullsized":"2024-09-21T07:26:22.755834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217328+0000","last_change":"2024-09-21T07:26:15.754185+0000","last_active":"2024-09-21T07:26:23.217328+0000","last_peered":"2024-09-21T07:26:23.217328+0000","last_clean":"2024-09-21T07:26:23.217328+0000","last_became_active":"2024-09-21T07:26:15.753790+0000","last_became_peered":"2024-09-21T07:26:15.753790+0000","last_unstale":"2024-09-21T07:26:23.217328+0000","last_undegraded":"2024-09-21T07:26:23.217328+0000","last_fullsized":"2024-09-21T07:26:23.217328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.769408+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.767688+0000","last_became_peered":"2024-09-21T07:26:15.767688+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:46.925801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":704845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":113985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":114021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":113895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":114055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":888832,"data_stored":704901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:26:37.192 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:26:37.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T07:26:37.192 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T07:26:37.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/1519168580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40ceb50 msgr2=0x7fc1b40cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/1519168580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40ceb50 0x7fc1b40cef60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc1b406e600 tx=0x7fc1ac02aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/1519168580 shutdown_connections 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/1519168580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40ceb50 0x7fc1b40cef60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/1519168580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40b3710 0x7fc1b406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.339+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/1519168580 >> 172.21.15.105:0/1519168580 conn(0x7fc1b409d850 msgr2=0x7fc1b408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.340+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/1519168580 shutdown_connections 2024-09-21T07:26:37.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.340+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/1519168580 wait complete. 2024-09-21T07:26:37.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.341+0000 7fc1bc3ba640 1 Processor -- start 2024-09-21T07:26:37.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.341+0000 7fc1bc3ba640 1 -- start start 2024-09-21T07:26:37.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.342+0000 7fc1bc3ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.342+0000 7fc1bc3ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 0x7fc1b41538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.342+0000 7fc1bc3ba640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40ceb50 0x7fc1b4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1bc3ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b40d0e70 con 0x7fc1b40b3710 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1bc3ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b40d0cf0 con 0x7fc1b40b9790 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1bc3ba640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b40d0ff0 con 0x7fc1b40ceb50 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59678/0 (socket says 172.21.15.105:59678) 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 -- 172.21.15.105:0/198975414 learned_addr learned my addr 172.21.15.105:0/198975414 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba957640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40ceb50 0x7fc1b4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1b9955640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 0x7fc1b41538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 -- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40ceb50 msgr2=0x7fc1b4153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40ceb50 0x7fc1b4153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 -- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 msgr2=0x7fc1b41538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 0x7fc1b41538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 -- 172.21.15.105:0/198975414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b40cdf30 con 0x7fc1b40b3710 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1b9955640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 0x7fc1b41538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1ba156640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c940 tx=0x7fc1b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0004220 con 0x7fc1b40b3710 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b00079b0 con 0x7fc1b40b3710 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b40c87f0 con 0x7fc1b40b3710 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0007e00 con 0x7fc1b40b3710 2024-09-21T07:26:37.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.343+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b4152f90 con 0x7fc1b40b3710 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.344+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b40d1530 con 0x7fc1b40b3710 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.345+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1b0005030 con 0x7fc1b40b3710 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.345+0000 7fc1aaffd640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 0x7fc19807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.345+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc1b009b540 con 0x7fc1b40b3710 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.346+0000 7fc1b9955640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 0x7fc19807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.346+0000 7fc1b9955640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 0x7fc19807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40d4090 tx=0x7fc1a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:37.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.350+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1b0014070 con 0x7fc1b40b3710 2024-09-21T07:26:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.526+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fc1b4151b30 con 0x7fc1b40b3710 2024-09-21T07:26:37.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.527+0000 7fc1aaffd640 1 -- 172.21.15.105:0/198975414 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fc1b0063650 con 0x7fc1b40b3710 2024-09-21T07:26:37.529 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 msgr2=0x7fc19807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 0x7fc19807b0d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40d4090 tx=0x7fc1a4009040 comp rx=0 tx=0).stop 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 msgr2=0x7fc1b40cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c940 tx=0x7fc1b000ce00 comp rx=0 tx=0).stop 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 shutdown_connections 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc198078bc0 0x7fc19807b0d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b9790 0x7fc1b41538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1b40ceb50 0x7fc1b4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 --2- 172.21.15.105:0/198975414 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1b40b3710 0x7fc1b40cd9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 >> 172.21.15.105:0/198975414 conn(0x7fc1b409d850 msgr2=0x7fc1b4068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 shutdown_connections 2024-09-21T07:26:37.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.529+0000 7fc1bc3ba640 1 -- 172.21.15.105:0/198975414 wait complete. 2024-09-21T07:26:37.542 INFO:teuthology.orchestra.run.smithi105.stdout:[{"version":31,"timestamp":"2024-09-21T07:26:03.306986+0000","name":"","changes":[{"name":"osd.5/osd_mclock_max_capacity_iops_ssd","new_value":"5941.555027"}]}] 2024-09-21T07:26:37.542 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T07:26:37.542 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T07:26:37.543 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T07:26:37.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T07:26:37.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 -- 172.21.15.105:0/1913394353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 msgr2=0x7fb49c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/1913394353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c074530 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb494009830 tx=0x7fb49402a510 comp rx=0 tx=0).stop 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 -- 172.21.15.105:0/1913394353 shutdown_connections 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/1913394353 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c074530 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/1913394353 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c06d4b0 0x7fb49c15c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 -- 172.21.15.105:0/1913394353 >> 172.21.15.105:0/1913394353 conn(0x7fb49c0cdce0 msgr2=0x7fb49c0ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 -- 172.21.15.105:0/1913394353 shutdown_connections 2024-09-21T07:26:37.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.653+0000 7fb4a32bd640 1 -- 172.21.15.105:0/1913394353 wait complete. 2024-09-21T07:26:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.655+0000 7fb4a32bd640 1 Processor -- start 2024-09-21T07:26:37.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.655+0000 7fb4a32bd640 1 -- start start 2024-09-21T07:26:37.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c0701e0 0x7fb49c1e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb49c1f90c0 0x7fb49c1f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c074bf0 con 0x7fb49c074120 2024-09-21T07:26:37.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c074a70 con 0x7fb49c1f90c0 2024-09-21T07:26:37.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a32bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb49c074d70 con 0x7fb49c0701e0 2024-09-21T07:26:37.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a0858640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.660 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a0858640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59702/0 (socket says 172.21.15.105:59702) 2024-09-21T07:26:37.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.656+0000 7fb4a0858640 1 -- 172.21.15.105:0/2257199044 learned_addr learned my addr 172.21.15.105:0/2257199044 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:37.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a1059640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c0701e0 0x7fb49c1e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.661 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a185a640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb49c1f90c0 0x7fb49c1f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 -- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c0701e0 msgr2=0x7fb49c1e9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c0701e0 0x7fb49c1e9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 -- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb49c1f90c0 msgr2=0x7fb49c1f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb49c1f90c0 0x7fb49c1f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 -- 172.21.15.105:0/2257199044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c0b26b0 con 0x7fb49c074120 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a0858640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c970 tx=0x7fb48c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c007bb0 con 0x7fb49c074120 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb48c007e40 con 0x7fb49c074120 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c0b2970 con 0x7fb49c074120 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c00f6b0 con 0x7fb49c074120 2024-09-21T07:26:37.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.657+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c0b2f30 con 0x7fb49c074120 2024-09-21T07:26:37.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.658+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb49c0d23b0 con 0x7fb49c074120 2024-09-21T07:26:37.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.659+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb48c00f850 con 0x7fb49c074120 2024-09-21T07:26:37.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.660+0000 7fb491ffb640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 0x7fb48007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:37.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.660+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb48c09b690 con 0x7fb49c074120 2024-09-21T07:26:37.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.660+0000 7fb4a1059640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 0x7fb48007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:37.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.660+0000 7fb4a1059640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 0x7fb48007b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4980017a0 tx=0x7fb498001730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:37.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.664+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb48c014030 con 0x7fb49c074120 2024-09-21T07:26:37.890 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.888+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb49c075220 con 0x7fb49c074120 2024-09-21T07:26:37.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.891+0000 7fb491ffb640 1 -- 172.21.15.105:0/2257199044 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194722 (secure 0 0 0) 0x7fb48c068e20 con 0x7fb49c074120 2024-09-21T07:26:37.895 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 msgr2=0x7fb48007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 0x7fb48007b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb4980017a0 tx=0x7fb498001730 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 msgr2=0x7fb49c1f8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00c970 tx=0x7fb48c00ce30 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 shutdown_connections 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb480078b90 0x7fb48007b0a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb49c1f90c0 0x7fb49c1f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb49c074120 0x7fb49c1f8b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 --2- 172.21.15.105:0/2257199044 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb49c0701e0 0x7fb49c1e9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 >> 172.21.15.105:0/2257199044 conn(0x7fb49c0cdce0 msgr2=0x7fb49c15e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 shutdown_connections 2024-09-21T07:26:37.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:37.894+0000 7fb4a32bd640 1 -- 172.21.15.105:0/2257199044 wait complete. 2024-09-21T07:26:37.907 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1954024229},{"type":"v1","addr":"172.21.15.112:6801","nonce":1954024229}]},"active_addr":"172.21.15.112:6801/1954024229","active_change":"2024-09-21T07:25:54.755610+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4114,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3749496995}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1737949316}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1978324892}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2921736159}]}]} 2024-09-21T07:26:37.912 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T07:26:37.912 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T07:26:37.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 -- 172.21.15.105:0/2576591796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6850 msgr2=0x7fedf00cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 --2- 172.21.15.105:0/2576591796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6850 0x7fedf00cf760 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fedf00ba400 tx=0x7fede802aa00 comp rx=0 tx=0).stop 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 -- 172.21.15.105:0/2576591796 shutdown_connections 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 --2- 172.21.15.105:0/2576591796 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6850 0x7fedf00cf760 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 -- 172.21.15.105:0/2576591796 >> 172.21.15.105:0/2576591796 conn(0x7fedf008bfb0 msgr2=0x7fedf00d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 -- 172.21.15.105:0/2576591796 shutdown_connections 2024-09-21T07:26:38.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.022+0000 7fedf7169640 1 -- 172.21.15.105:0/2576591796 wait complete. 2024-09-21T07:26:38.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.023+0000 7fedf7169640 1 Processor -- start 2024-09-21T07:26:38.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.024+0000 7fedf7169640 1 -- start start 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf008c3c0 0x7fedf0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf00b6850 0x7fedf0152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fedf0169f30 con 0x7fedf00b6440 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fedf00b3b60 con 0x7fedf00b6850 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf7169640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fedf016a0b0 con 0x7fedf008c3c0 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59726/0 (socket says 172.21.15.105:59726) 2024-09-21T07:26:38.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 -- 172.21.15.105:0/3587261228 learned_addr learned my addr 172.21.15.105:0/3587261228 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf5706640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf00b6850 0x7fedf0152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedf4f05640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf008c3c0 0x7fedf0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 -- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf008c3c0 msgr2=0x7fedf0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf008c3c0 0x7fedf0154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 -- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf00b6850 msgr2=0x7fedf0152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf00b6850 0x7fedf0152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.025+0000 7fedeffff640 1 -- 172.21.15.105:0/3587261228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf0151780 con 0x7fedf00b6440 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7fedeffff640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fede400c3c0 tx=0x7fede400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4017070 con 0x7fedf00b6440 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fede40035f0 con 0x7fedf00b6440 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf015cbd0 con 0x7fedf00b6440 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4003a80 con 0x7fedf00b6440 2024-09-21T07:26:38.029 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.026+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf00b98c0 con 0x7fedf00b6440 2024-09-21T07:26:38.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.027+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf006f7f0 con 0x7fedf00b6440 2024-09-21T07:26:38.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.027+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fede401a070 con 0x7fedf00b6440 2024-09-21T07:26:38.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.028+0000 7feded7fa640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 0x7feddc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.028+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fede409a3d0 con 0x7fedf00b6440 2024-09-21T07:26:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.028+0000 7fedf4f05640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 0x7feddc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.031 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.028+0000 7fedf4f05640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 0x7feddc07b0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fede0006fd0 tx=0x7fede0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.034 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.032+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedf00cfca0 con 0x7fedf00b6440 2024-09-21T07:26:38.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.210+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedf0151510 con 0x7fedf00b6440 2024-09-21T07:26:38.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.211+0000 7feded7fa640 1 -- 172.21.15.105:0/3587261228 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7fede40624d0 con 0x7fedf00b6440 2024-09-21T07:26:38.213 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:38.213 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 msgr2=0x7feddc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 0x7feddc07b0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fede0006fd0 tx=0x7fede0008040 comp rx=0 tx=0).stop 2024-09-21T07:26:38.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 msgr2=0x7fedf014f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fede400c3c0 tx=0x7fede400c880 comp rx=0 tx=0).stop 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 shutdown_connections 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7feddc078bc0 0x7feddc07b0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf00b6850 0x7fedf0152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fedf00b6440 0x7fedf014f720 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 --2- 172.21.15.105:0/3587261228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf008c3c0 0x7fedf0154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 >> 172.21.15.105:0/3587261228 conn(0x7fedf008bfb0 msgr2=0x7fedf00cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 shutdown_connections 2024-09-21T07:26:38.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.213+0000 7fedf7169640 1 -- 172.21.15.105:0/3587261228 wait complete. 2024-09-21T07:26:38.227 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T07:26:38.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:38.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 -- 172.21.15.105:0/3892676461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 msgr2=0x7f1db8167070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/3892676461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db8167070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009830 tx=0x7f1db002a510 comp rx=0 tx=0).stop 2024-09-21T07:26:38.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 -- 172.21.15.105:0/3892676461 shutdown_connections 2024-09-21T07:26:38.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/3892676461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db8167070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/3892676461 >> v2:172.21.15.112:3300/0 conn(0x7f1db80cff70 0x7f1db80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 -- 172.21.15.105:0/3892676461 >> 172.21.15.105:0/3892676461 conn(0x7f1db80cee20 msgr2=0x7f1db80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 -- 172.21.15.105:0/3892676461 shutdown_connections 2024-09-21T07:26:38.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.338+0000 7f1dbe24d640 1 -- 172.21.15.105:0/3892676461 wait complete. 2024-09-21T07:26:38.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.340+0000 7f1dbe24d640 1 Processor -- start 2024-09-21T07:26:38.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.340+0000 7f1dbe24d640 1 -- start start 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db81e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db80cff70 0x7f1db81cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1db80b77d0 con 0x7f1db80cd090 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8074af0 con 0x7f1db80cff70 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1dbe24d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1db80b7950 con 0x7f1db80cdbb0 2024-09-21T07:26:38.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59740/0 (socket says 172.21.15.105:59740) 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1db77fe640 1 -- 172.21.15.105:0/961348617 learned_addr learned my addr 172.21.15.105:0/961348617 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1db6ffd640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db81e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.341+0000 7f1db7fff640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db80cff70 0x7f1db81cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 -- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 msgr2=0x7f1db81e8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db81e8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 -- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db80cff70 msgr2=0x7f1db81cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db80cff70 0x7f1db81cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 -- 172.21.15.105:0/961348617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db81cc380 con 0x7f1db80cd090 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1db77fe640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1da800ecb0 tx=0x7f1da800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8003b70 con 0x7f1db80cd090 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1da8003d10 con 0x7f1db80cd090 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db81cc0a0 con 0x7f1db80cd090 2024-09-21T07:26:38.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8004280 con 0x7f1db80cd090 2024-09-21T07:26:38.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.342+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81f72e0 con 0x7f1db80cd090 2024-09-21T07:26:38.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.343+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db809bb10 con 0x7f1db80cd090 2024-09-21T07:26:38.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.344+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1da8004420 con 0x7f1db80cd090 2024-09-21T07:26:38.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.344+0000 7f1d9ffff640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 0x7f1da407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.345+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1da809bc20 con 0x7f1db80cd090 2024-09-21T07:26:38.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.345+0000 7f1db6ffd640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 0x7f1da407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.345+0000 7f1db6ffd640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 0x7f1da407b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1dac007d60 tx=0x7f1dac007cf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.349+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1da80636e0 con 0x7f1db80cd090 2024-09-21T07:26:38.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.527+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1db80b81d0 con 0x7f1db80cd090 2024-09-21T07:26:38.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.528+0000 7f1d9ffff640 1 -- 172.21.15.105:0/961348617 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f1da8068110 con 0x7f1db80cd090 2024-09-21T07:26:38.531 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:38.531 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.530+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 msgr2=0x7f1da407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 0x7f1da407b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1dac007d60 tx=0x7f1dac007cf0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 msgr2=0x7f1db81f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1da800ecb0 tx=0x7f1da800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 shutdown_connections 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1da4078b90 0x7f1da407b0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1db80cff70 0x7f1db81cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1db80cdbb0 0x7f1db81e8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 --2- 172.21.15.105:0/961348617 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1db80cd090 0x7f1db81f7ae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 >> 172.21.15.105:0/961348617 conn(0x7f1db80cee20 msgr2=0x7f1db81e53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 shutdown_connections 2024-09-21T07:26:38.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.531+0000 7f1dbe24d640 1 -- 172.21.15.105:0/961348617 wait complete. 2024-09-21T07:26:38.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T07:26:38.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T07:26:38.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T07:26:38.545 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T07:26:38.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T07:26:38.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T07:26:38.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T07:26:38.547 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T07:26:38.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 -- 172.21.15.105:0/173729341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e00ce460 msgr2=0x7fa9e0158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/173729341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e00ce460 0x7fa9e0158630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d802bb60 comp rx=0 tx=0).stop 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 -- 172.21.15.105:0/173729341 shutdown_connections 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/173729341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/173729341 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e00ce460 0x7fa9e0158630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 -- 172.21.15.105:0/173729341 >> 172.21.15.105:0/173729341 conn(0x7fa9e0073660 msgr2=0x7fa9e0073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 -- 172.21.15.105:0/173729341 shutdown_connections 2024-09-21T07:26:38.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.757+0000 7fa9e5a39640 1 -- 172.21.15.105:0/173729341 wait complete. 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.758+0000 7f9123fa8640 1 -- 172.21.15.105:0/2092118961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 msgr2=0x7f911c160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.758+0000 7f9123fa8640 1 --2- 172.21.15.105:0/2092118961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c160aa0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9118005fd0 tx=0x7f911802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 -- 172.21.15.105:0/2092118961 shutdown_connections 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 --2- 172.21.15.105:0/2092118961 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 0x7f911c0704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 --2- 172.21.15.105:0/2092118961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c160aa0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 -- 172.21.15.105:0/2092118961 >> 172.21.15.105:0/2092118961 conn(0x7f911c0cff40 msgr2=0x7f911c0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7fdde959a640 1 -- 172.21.15.105:0/3848615194 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdddc036400 con 0x7fddec0cf3b0 2024-09-21T07:26:38.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 -- 172.21.15.105:0/2092118961 shutdown_connections 2024-09-21T07:26:38.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7f9123fa8640 1 -- 172.21.15.105:0/2092118961 wait complete. 2024-09-21T07:26:38.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7fddf0d58640 1 -- 172.21.15.105:0/3848615194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf3b0 msgr2=0x7fddec15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.759+0000 7fddf0d58640 1 --2- 172.21.15.105:0/3848615194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf3b0 0x7fddec15e980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005fd0 tx=0x7fdddc02aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:38.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fddf0d58640 1 -- 172.21.15.105:0/3848615194 shutdown_connections 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fddf0d58640 1 --2- 172.21.15.105:0/3848615194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf3b0 0x7fddec15e980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fddf0d58640 1 --2- 172.21.15.105:0/3848615194 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec140ff0 0x7fddec13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fddf0d58640 1 -- 172.21.15.105:0/3848615194 >> 172.21.15.105:0/3848615194 conn(0x7fddec06cda0 msgr2=0x7fddec06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fddf0d58640 1 -- 172.21.15.105:0/3848615194 shutdown_connections 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 -- 172.21.15.105:0/764893582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c00bc30 msgr2=0x7f220c0df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 --2- 172.21.15.105:0/764893582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c00bc30 0x7f220c0df180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2210009820 tx=0x7f221002a500 comp rx=0 tx=0).stop 2024-09-21T07:26:38.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fa9e5a39640 1 Processor -- start 2024-09-21T07:26:38.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 -- 172.21.15.105:0/764893582 shutdown_connections 2024-09-21T07:26:38.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7fa9e5a39640 1 -- start start 2024-09-21T07:26:38.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 --2- 172.21.15.105:0/764893582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 0x7f220c005c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 --2- 172.21.15.105:0/764893582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c00bc30 0x7f220c0df180 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2210009820 tx=0x7f221002a500 comp rx=0 tx=0).stop 2024-09-21T07:26:38.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.760+0000 7f221aae1640 1 -- 172.21.15.105:0/764893582 >> 172.21.15.105:0/764893582 conn(0x7f220c051420 msgr2=0x7f220c051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7f2ff5dad640 1 -- 172.21.15.105:0/2746145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00cb110 msgr2=0x7f2ff006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/2746145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00cb110 0x7f2ff006ef50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8005fd0 tx=0x7f2fe802aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:38.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7f221aae1640 1 -- 172.21.15.105:0/764893582 shutdown_connections 2024-09-21T07:26:38.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7f221aae1640 1 -- 172.21.15.105:0/764893582 wait complete. 2024-09-21T07:26:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7f2ff5dad640 1 -- 172.21.15.105:0/2746145020 shutdown_connections 2024-09-21T07:26:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/2746145020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff006f490 0x7f2ff006f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/2746145020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff00cb110 0x7f2ff006ef50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f2ff5dad640 1 -- 172.21.15.105:0/2746145020 >> 172.21.15.105:0/2746145020 conn(0x7f2ff00ccb90 msgr2=0x7f2ff00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.768 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f2ff5dad640 1 -- 172.21.15.105:0/2746145020 shutdown_connections 2024-09-21T07:26:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f2ff5dad640 1 -- 172.21.15.105:0/2746145020 wait complete. 2024-09-21T07:26:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.763+0000 7fa9e5a39640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e01f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.763+0000 7fa9e5a39640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e0070c10 0x7fa9e0150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.763+0000 7fa9e5a39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e00ce460 0x7fa9e00b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f9123fa8640 1 Processor -- start 2024-09-21T07:26:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.762+0000 7f9123fa8640 1 -- start start 2024-09-21T07:26:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.763+0000 7fa9e4a37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e01f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9e4a37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e01f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38278/0 (socket says 172.21.15.105:38278) 2024-09-21T07:26:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9e5a39640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0069b30 con 0x7fa9e0070c10 2024-09-21T07:26:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.761+0000 7fddf0d58640 1 -- 172.21.15.105:0/3848615194 wait complete. 2024-09-21T07:26:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fddf0d58640 1 Processor -- start 2024-09-21T07:26:38.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f221aae1640 1 Processor -- start 2024-09-21T07:26:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e0070c10 0x7fa9e0150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fddf0d58640 1 -- start start 2024-09-21T07:26:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e0070c10 0x7fa9e0150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59766/0 (socket says 172.21.15.105:59766) 2024-09-21T07:26:38.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9dffff640 1 -- 172.21.15.105:0/2416984211 learned_addr learned my addr 172.21.15.105:0/2416984211 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f221aae1640 1 -- start start 2024-09-21T07:26:38.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e00699b0 con 0x7fa9e00ce460 2024-09-21T07:26:38.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e0069cb0 con 0x7fa9e006ddb0 2024-09-21T07:26:38.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 0x7f911c14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c15c650 0x7f911c149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7fa9e5238640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e00ce460 0x7fa9e00b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0cf0c0 con 0x7f911c15c650 2024-09-21T07:26:38.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f911c0cef40 con 0x7f911c073450 2024-09-21T07:26:38.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9123fa8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f911c0cf240 con 0x7f911c070090 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9121d44640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 0x7f911c14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9121d44640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 0x7f911c14c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38280/0 (socket says 172.21.15.105:38280) 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9121d44640 1 -- 172.21.15.105:0/2711548842 learned_addr learned my addr 172.21.15.105:0/2711548842 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.764+0000 7f9121543640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 -- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e00ce460 msgr2=0x7fa9e00b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9e00ce460 0x7fa9e00b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 -- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e0070c10 msgr2=0x7fa9e0150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9e0070c10 0x7fa9e0150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e01f4260 con 0x7fa9e006ddb0 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9122545640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c15c650 0x7f911c149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 -- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 msgr2=0x7f911c14c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f911c070090 0x7f911c14c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 -- 172.21.15.105:0/2711548842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c15c650 msgr2=0x7f911c149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c15c650 0x7f911c149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c1e6840 con 0x7f911c073450 2024-09-21T07:26:38.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9122545640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c15c650 0x7f911c149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7fa9e4a37640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e01f3d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800a000 tx=0x7fa9d8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 -- 172.21.15.105:0/1947418751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 msgr2=0x7f816c053aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 --2- 172.21.15.105:0/1947418751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c053aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f81740666e0 tx=0x7f81740a66c0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 -- 172.21.15.105:0/1947418751 shutdown_connections 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 --2- 172.21.15.105:0/1947418751 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c053aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 --2- 172.21.15.105:0/1947418751 >> v2:172.21.15.112:3300/0 conn(0x7f816c0e3200 0x7f816c053150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 --2- 172.21.15.105:0/1947418751 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 0x7f816c007550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f817a561640 1 -- 172.21.15.105:0/1947418751 >> 172.21.15.105:0/1947418751 conn(0x7f816c03a680 msgr2=0x7f816c03aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.765+0000 7f9121543640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c149520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c370 tx=0x7f910c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004450 con 0x7fa9e006ddb0 2024-09-21T07:26:38.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f2ff5dad640 1 Processor -- start 2024-09-21T07:26:38.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d8041070 con 0x7fa9e006ddb0 2024-09-21T07:26:38.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e0152e30 con 0x7fa9e006ddb0 2024-09-21T07:26:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f817a561640 1 -- 172.21.15.105:0/1947418751 shutdown_connections 2024-09-21T07:26:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e0150180 con 0x7fa9e006ddb0 2024-09-21T07:26:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 -- 172.21.15.105:0/3730546384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 msgr2=0x7f1f780e3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 --2- 172.21.15.105:0/3730546384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 0x7f1f780e3190 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006c0c0 tx=0x7f1f800a8490 comp rx=0 tx=0).stop 2024-09-21T07:26:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f2ff5dad640 1 -- start start 2024-09-21T07:26:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f817a561640 1 -- 172.21.15.105:0/1947418751 wait complete. 2024-09-21T07:26:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f221aae1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220c009ba0 0x7f220c0d8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c015e30 con 0x7f911c073450 2024-09-21T07:26:38.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f910c0050d0 con 0x7f911c073450 2024-09-21T07:26:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c0035f0 con 0x7f911c073450 2024-09-21T07:26:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f9123fa8640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c1e6ab0 con 0x7f911c073450 2024-09-21T07:26:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f221aae1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 0x7f220c021dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fddf0d58640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 0x7fddec1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f221aae1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c0e1cc0 0x7f220c0e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f221aae1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f220c006aa0 con 0x7f220c009ba0 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 -- 172.21.15.105:0/3730546384 shutdown_connections 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 --2- 172.21.15.105:0/3730546384 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 0x7f1f780e3190 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 --2- 172.21.15.105:0/3730546384 >> v2:172.21.15.105:3300/0 conn(0x7f1f780df390 0x7f1f78007630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f1f87440640 1 -- 172.21.15.105:0/3730546384 >> 172.21.15.105:0/3730546384 conn(0x7f1f7801ddf0 msgr2=0x7f1f78039ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fddf0d58640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf7c0 0x7fddec201ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f9123fa8640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1e7070 con 0x7f911c073450 2024-09-21T07:26:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8003e50 con 0x7fa9e006ddb0 2024-09-21T07:26:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fddea59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 0x7fddec1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f22192de640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 0x7f220c021dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7f22192de640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 0x7f220c021dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38290/0 (socket says 172.21.15.105:38290) 2024-09-21T07:26:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f22192de640 1 -- 172.21.15.105:0/4066364569 learned_addr learned my addr 172.21.15.105:0/4066364569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f1f87440640 1 -- 172.21.15.105:0/3730546384 shutdown_connections 2024-09-21T07:26:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f220c006920 con 0x7f220c0e1cc0 2024-09-21T07:26:38.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f220c006c20 con 0x7f220c036b10 2024-09-21T07:26:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f1f87440640 1 -- 172.21.15.105:0/3730546384 wait complete. 2024-09-21T07:26:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f2219adf640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220c009ba0 0x7f220c0d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221a2e0640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c0e1cc0 0x7f220c0e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f76857c3640 1 -- 172.21.15.105:0/256050906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76780df390 msgr2=0x7f7678007630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f76857c3640 1 --2- 172.21.15.105:0/256050906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76780df390 0x7f7678007630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f766c00b9b0 tx=0x7f766c02c130 comp rx=0 tx=0).stop 2024-09-21T07:26:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fddea59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 0x7fddec1575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38304/0 (socket says 172.21.15.105:38304) 2024-09-21T07:26:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.766+0000 7fddea59c640 1 -- 172.21.15.105:0/2234476160 learned_addr learned my addr 172.21.15.105:0/2234476160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7fddf0d58640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec140ff0 0x7fddec0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7fddf0d58640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fddec15f250 con 0x7fddec140ff0 2024-09-21T07:26:38.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7fddf0d58640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fddec15f0d0 con 0x7fddec0cf7c0 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7fddf0d58640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fddec0696c0 con 0x7fddec0cf3b0 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7fdde9d9b640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf7c0 0x7fddec201ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec140ff0 0x7fddec0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 -- 172.21.15.105:0/256050906 shutdown_connections 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 --2- 172.21.15.105:0/256050906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7678054b70 0x7f7678054f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 --2- 172.21.15.105:0/256050906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76780df390 0x7f7678007630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 -- 172.21.15.105:0/256050906 >> 172.21.15.105:0/256050906 conn(0x7f767801ddf0 msgr2=0x7f7678039ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 -- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 msgr2=0x7fddec1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 0x7fddec1575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 -- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf7c0 msgr2=0x7fddec201ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf7c0 0x7fddec201ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddead9d640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec1588c0 con 0x7fddec140ff0 2024-09-21T07:26:38.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 -- 172.21.15.105:0/256050906 shutdown_connections 2024-09-21T07:26:38.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fddea59c640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fddec0cf3b0 0x7fddec1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:38.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f2ff5dad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ff006f490 0x7f2ff0154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fdde9d9b640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fddec0cf7c0 0x7fddec201ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:38.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9e006e1c0 con 0x7fa9e006ddb0 2024-09-21T07:26:38.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2ff5dad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 0x7f2ff014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f9123fa8640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9104001030 con 0x7f911c073450 2024-09-21T07:26:38.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2ff5dad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0150200 0x7f2ff01506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f817a561640 1 Processor -- start 2024-09-21T07:26:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 0x7f2ff014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2ff5dad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff00701a0 con 0x7f2ff006f490 2024-09-21T07:26:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2feeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 0x7f2ff014fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38320/0 (socket says 172.21.15.105:38320) 2024-09-21T07:26:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.768+0000 7f76857c3640 1 -- 172.21.15.105:0/256050906 wait complete. 2024-09-21T07:26:38.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2feeffd640 1 -- 172.21.15.105:0/161775840 learned_addr learned my addr 172.21.15.105:0/161775840 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221a2e0640 1 -- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 msgr2=0x7f220c021dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221a2e0640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f220c036b10 0x7f220c021dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221a2e0640 1 -- 172.21.15.105:0/4066364569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220c009ba0 msgr2=0x7f220c0d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.767+0000 7f221a2e0640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f220c009ba0 0x7f220c0d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f221a2e0640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c0e2610 con 0x7f220c0e1cc0 2024-09-21T07:26:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2fef7fe640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ff006f490 0x7f2ff0154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2feffff640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0150200 0x7f2ff01506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2ff5dad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0070020 con 0x7f2ff0150200 2024-09-21T07:26:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff0168e70 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7fddead9d640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec140ff0 0x7fddec0b1d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c390 tx=0x7fddd400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f817a561640 1 -- start start 2024-09-21T07:26:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f221a2e0640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c0e1cc0 0x7f220c0e20d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f221406a320 tx=0x7f221406a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 -- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0150200 msgr2=0x7f2ff01506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ff0150200 0x7f2ff01506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 -- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ff006f490 msgr2=0x7f2ff0154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4017070 con 0x7fddec140ff0 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f1f87440640 1 Processor -- start 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ff006f490 0x7f2ff0154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff01f96d0 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2fef7fe640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ff006f490 0x7f2ff0154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:38.795 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2feeffd640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 0x7f2ff014fcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009bf0 tx=0x7f2fe80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddd40053e0 con 0x7fddec140ff0 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7fddf0d58640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec157c90 con 0x7fddec140ff0 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7fddf0d58640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec1523e0 con 0x7fddec140ff0 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.769+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f910c003790 con 0x7f911c073450 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140763a0 con 0x7f220c0e1cc0 2024-09-21T07:26:38.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c021540 con 0x7f220c0e1cc0 2024-09-21T07:26:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c020780 con 0x7f220c0e1cc0 2024-09-21T07:26:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.770+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8045070 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f91127fc640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9100078c90 0x7f910007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9d8068d10 con 0x7fa9e006ddb0 2024-09-21T07:26:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fe8036600 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff0154360 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f9121d44640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9100078c90 0x7f910007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.798 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff0152be0 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f910c0a2b40 con 0x7f911c073450 2024-09-21T07:26:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8036a30 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7fa9dd7fa640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa9cc078e00 0x7fa9cc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f76857c3640 1 Processor -- start 2024-09-21T07:26:38.799 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa9d80c0250 con 0x7fa9e006ddb0 2024-09-21T07:26:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c179f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fa9dd7fa640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7fa9cc0825d0 0x7fa9cc084a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 0x7f816c0d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9d800c1c0 con 0x7fa9cc0825d0 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c0e3200 0x7f816c03d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0e71d0 con 0x7f816c053690 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f816c0e7050 con 0x7f816c0e3200 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817a561640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f816c0e7350 con 0x7f816c054190 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa9d80c0640 con 0x7fa9e006ddb0 2024-09-21T07:26:38.801 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fa9dffff640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa9cc078e00 0x7fa9cc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f9121d44640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9100078c90 0x7f910007b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9118000c00 tx=0x7f911800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817955f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c179f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817955f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c179f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59828/0 (socket says 172.21.15.105:59828) 2024-09-21T07:26:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f817955f640 1 -- 172.21.15.105:0/3191902962 learned_addr learned my addr 172.21.15.105:0/3191902962 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.802 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f8179d60640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c0e3200 0x7f816c03d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.771+0000 7f1f87440640 1 -- start start 2024-09-21T07:26:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f91127fc640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f9100082450 0x7f91000848e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.803 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2214076540 con 0x7f220c0e1cc0 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9100084fb0 con 0x7f9100082450 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f910c0a72a0 con 0x7f911c073450 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f76857c3640 1 -- start start 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 0x7f1f7817e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780df390 0x7f1f7801ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.804 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 0x7f1f781781d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7fa9e5238640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7fa9cc0825d0 0x7fa9cc084a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 0x7f816c0d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7803a340 con 0x7f1f78054b70 2024-09-21T07:26:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7803a1c0 con 0x7f1f780df390 2024-09-21T07:26:38.805 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f1f87440640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1f78004b40 con 0x7f1f780e2d80 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7fa9dffff640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa9cc078e00 0x7fa9cc07b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00b2550 tx=0x7fa9d0007f20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214075cf0 con 0x7f220c0e1cc0 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 -- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c0e3200 msgr2=0x7f816c03d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f816c0e3200 0x7f816c03d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 -- 172.21.15.105:0/3191902962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 msgr2=0x7f816c179f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f816c053690 0x7f816c179f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.806 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f8178d5e640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c0d7d90 con 0x7f816c054190 2024-09-21T07:26:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7fa9e5238640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7fa9cc0825d0 0x7fa9cc084a60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fd0001030 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fe8036bd0 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.807 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f21fffff640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f21f4001030 con 0x7f220c0e1cc0 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.773+0000 7f9122545640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f9100082450 0x7f91000848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fddd0001030 con 0x7fddec140ff0 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.772+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4003850 con 0x7fddec140ff0 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2fd7fff640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2fdc078bc0 0x7f2fdc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f1f8643e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 0x7f1f7817e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f1f8643e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 0x7f1f7817e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59832/0 (socket says 172.21.15.105:59832) 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f1f8643e640 1 -- 172.21.15.105:0/336293575 learned_addr learned my addr 172.21.15.105:0/336293575 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f8178d5e640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 0x7f816c0d75d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f816400d6b0 tx=0x7f816400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164017070 con 0x7f816c054190 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8164005310 con 0x7f816c054190 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 -- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 msgr2=0x7f1f781781d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f780e2d80 0x7f1f781781d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 -- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780df390 msgr2=0x7f1f7801ea40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f780df390 0x7f1f7801ea40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.809 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f78020fc0 con 0x7f1f78054b70 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2214094440 con 0x7f220c0e1cc0 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2fe80bc840 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2fd7fff640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f2fdc082380 0x7f2fdc084810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.774+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fdc084ee0 con 0x7f2fdc082380 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2fe80c1200 con 0x7f2ff00ce1f0 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f2fef7fe640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2fdc078bc0 0x7f2fdc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.810 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f2feffff640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f2fdc082380 0x7f2fdc084810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640047a0 con 0x7f816c054190 2024-09-21T07:26:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f220a7fc640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f21f8078c10 0x7f21f807b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f22141172d0 con 0x7f220c0e1cc0 2024-09-21T07:26:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c04fbd0 con 0x7f816c054190 2024-09-21T07:26:38.811 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f2219adf640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f21f8078c10 0x7f21f807b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f1f8643e640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 0x7f1f7817e660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0078f0 tx=0x7f1f7c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7f9122545640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f9100082450 0x7f91000848e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== osd.4 v2:172.21.15.112:6810/4146883683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9100084fb0 con 0x7f9100082450 2024-09-21T07:26:38.812 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f2feffff640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f2fdc082380 0x7f2fdc084810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c050190 con 0x7f816c054190 2024-09-21T07:26:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.775+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== osd.3 v2:172.21.15.105:6800/2773911429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9d800c1c0 con 0x7fa9cc0825d0 2024-09-21T07:26:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 0x7f7678053aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.813 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018030 con 0x7f1f78054b70 2024-09-21T07:26:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f220a7fc640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f21f8082350 0x7f21f80847e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7678054b70 0x7f76780316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f22192de640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f21f8082350 0x7f21f80847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.814 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f2fef7fe640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2fdc078bc0 0x7f2fdc07b0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0001400 tx=0x7f2fe0016040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fddd401a070 con 0x7fddec140ff0 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde2ffd640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fddcc078c90 0x7fddcc07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fddd409a980 con 0x7fddec140ff0 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f780212a0 con 0x7f1f78054b70 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f78021860 con 0x7f1f78054b70 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f7c004110 con 0x7f1f78054b70 2024-09-21T07:26:38.815 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00e920 con 0x7f1f78054b70 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f221408fe50 con 0x7f21f8082350 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f22192de640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f21f8082350 0x7f21f80847e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde2ffd640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7fddcc082460 0x7fddcc0848f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddd4009e50 con 0x7fddcc082460 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fddd409ad70 con 0x7fddec140ff0 2024-09-21T07:26:38.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f2219adf640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f21f8078c10 0x7f21f807b120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2210034000 tx=0x7f2210004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f22140db400 con 0x7f220c0e1cc0 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f767ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 0x7f7678053aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f767ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 0x7f7678053aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59846/0 (socket says 172.21.15.105:59846) 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76780df390 0x7f7678031c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7678034140 con 0x7f7678006050 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f767803a790 con 0x7f7678054b70 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f76857c3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f767803a1c0 con 0x7f76780df390 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.776+0000 7f767ffff640 1 -- 172.21.15.105:0/1654436320 learned_addr learned my addr 172.21.15.105:0/1654436320 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767f7fe640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7678054b70 0x7f76780316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f7684fc2640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76780df390 0x7f7678031c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767ffff640 1 -- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76780df390 msgr2=0x7f7678031c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.817 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767ffff640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76780df390 0x7f7678031c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767ffff640 1 -- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7678054b70 msgr2=0x7f76780316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767ffff640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7678054b70 0x7f76780316e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f767ffff640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780323f0 con 0x7f7678006050 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== osd.0 v2:172.21.15.105:6824/2340388403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fdc084ee0 con 0x7f2fdc082380 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f7684fc2640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76780df390 0x7f7678031c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:38.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f816c0dfb10 con 0x7f816c054190 2024-09-21T07:26:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.778+0000 7f767ffff640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 0x7f7678053aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bad0 tx=0x7f766c035ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.777+0000 7fdde9d9b640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7fddcc082460 0x7fddcc0848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.778+0000 7fddea59c640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fddcc078c90 0x7fddcc07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.819 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.778+0000 7fdde9d9b640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7fddcc082460 0x7fddcc0848f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.778+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f68001030 con 0x7f1f78054b70 2024-09-21T07:26:38.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.779+0000 7fddea59c640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fddcc078c90 0x7fddcc07b1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdde400abe0 tx=0x7fdde4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.779+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== osd.5 v2:172.21.15.112:6826/2151662407 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f221408fe50 con 0x7f21f8082350 2024-09-21T07:26:38.820 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.779+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8164004940 con 0x7f816c054190 2024-09-21T07:26:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.779+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c043070 con 0x7f7678006050 2024-09-21T07:26:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.779+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f766c035d10 con 0x7f7678006050 2024-09-21T07:26:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.780+0000 7f8169ffb640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8158078c60 0x7f815807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.780+0000 7f817955f640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8158078c60 0x7f815807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.821 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.780+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== osd.2 v2:172.21.15.105:6808/3103137694 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddd4009e50 con 0x7fddcc082460 2024-09-21T07:26:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.780+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c039040 con 0x7f7678006050 2024-09-21T07:26:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7678181f20 con 0x7f7678006050 2024-09-21T07:26:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.780+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f7c01b070 con 0x7f1f78054b70 2024-09-21T07:26:38.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f816409ae40 con 0x7f816c054190 2024-09-21T07:26:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f817955f640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8158078c60 0x7f815807b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81700058c0 tx=0x7f8170009260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7678053fe0 con 0x7f7678006050 2024-09-21T07:26:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f8169ffb640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f8158082430 0x7f81580848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f1f76ffd640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1f64078c60 0x7f1f6407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8158084f90 con 0x7f8158082430 2024-09-21T07:26:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1f7c09ba80 con 0x7f1f78054b70 2024-09-21T07:26:38.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f816409d4a0 con 0x7f816c054190 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f1f76ffd640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f1f64082430 0x7f1f640848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f64084f90 con 0x7f1f64082430 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.781+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c09be70 con 0x7f1f78054b70 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.782+0000 7f1f85c3d640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1f64078c60 0x7f1f6407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.782+0000 7f1f86c3f640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f1f64082430 0x7f1f640848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.782+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76781825e0 con 0x7f7678006050 2024-09-21T07:26:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.782+0000 7f8179d60640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f8158082430 0x7f81580848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.783+0000 7f8179d60640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f8158082430 0x7f81580848c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.782+0000 7f1f86c3f640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f1f64082430 0x7f1f640848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.783+0000 7f1f85c3d640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1f64078c60 0x7f1f6407b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f780dfcb0 tx=0x7f1f70007970 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.784+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== osd.1 v2:172.21.15.105:6816/4277636459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f64084f90 con 0x7f1f64082430 2024-09-21T07:26:38.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.784+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== osd.7 v2:172.21.15.112:6802/3904329758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8158084f90 con 0x7f8158082430 2024-09-21T07:26:38.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.784+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f766c046070 con 0x7f7678006050 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.785+0000 7f767cff9640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7664078c60 0x7f766407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.785+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f766c0bb6d0 con 0x7f7678006050 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.785+0000 7f767f7fe640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7664078c60 0x7f766407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.786+0000 7f767cff9640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f7664082430 0x7f76640848c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.786+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f766c005640 con 0x7f7664082430 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.786+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f766c0bbac0 con 0x7f7678006050 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.787+0000 7f7684fc2640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f7664082430 0x7f76640848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.787+0000 7f767f7fe640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7664078c60 0x7f766407b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7678054950 tx=0x7f7674005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.787+0000 7f7684fc2640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f7664082430 0x7f76640848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:38.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.788+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== osd.6 v2:172.21.15.112:6818/2596982695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f766c005640 con 0x7f7664082430 2024-09-21T07:26:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.805+0000 7f9123fa8640 1 -- 172.21.15.105:0/2711548842 --> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9104002dd0 con 0x7f9100082450 2024-09-21T07:26:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.806+0000 7f91127fc640 1 -- 172.21.15.105:0/2711548842 <== osd.4 v2:172.21.15.112:6810/4146883683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9104002dd0 con 0x7f9100082450 2024-09-21T07:26:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.806+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 --> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9e0069360 con 0x7fa9cc0825d0 2024-09-21T07:26:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.806+0000 7fa9dd7fa640 1 -- 172.21.15.105:0/2416984211 <== osd.3 v2:172.21.15.105:6800/2773911429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9e0069360 con 0x7fa9cc0825d0 2024-09-21T07:26:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7fa9cc0825d0 msgr2=0x7fa9cc084a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f9100082450 msgr2=0x7f91000848e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7f90fffff640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f9100082450 0x7f91000848e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7fa9cc0825d0 0x7fa9cc084a60 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9100078c90 msgr2=0x7f910007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7f90fffff640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9100078c90 0x7f910007b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9118000c00 tx=0x7f911800b040 comp rx=0 tx=0).stop 2024-09-21T07:26:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa9cc078e00 msgr2=0x7fa9cc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.807+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 --> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fd0002dd0 con 0x7f2fdc082380 2024-09-21T07:26:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 msgr2=0x7f911c149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 --2- 172.21.15.105:0/2711548842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f911c073450 0x7f911c149520 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f910c00c370 tx=0x7f910c00c830 comp rx=0 tx=0).stop 2024-09-21T07:26:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f9122545640 1 -- 172.21.15.105:0/2711548842 reap_dead start 2024-09-21T07:26:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa9cc078e00 0x7fa9cc07b310 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00b2550 tx=0x7fa9d0007f20 comp rx=0 tx=0).stop 2024-09-21T07:26:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 msgr2=0x7fa9e01f3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5a39640 1 --2- 172.21.15.105:0/2416984211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9e006ddb0 0x7fa9e01f3d20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800a000 tx=0x7fa9d8003ba0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 shutdown_connections 2024-09-21T07:26:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 >> 172.21.15.105:0/2711548842 conn(0x7f911c0cff40 msgr2=0x7f911c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f2fd7fff640 1 -- 172.21.15.105:0/161775840 <== osd.0 v2:172.21.15.105:6824/2340388403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2fd0002dd0 con 0x7f2fdc082380 2024-09-21T07:26:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 shutdown_connections 2024-09-21T07:26:38.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7f90fffff640 1 -- 172.21.15.105:0/2711548842 wait complete. 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5238640 1 -- 172.21.15.105:0/2416984211 reap_dead start 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 shutdown_connections 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.808+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 >> 172.21.15.105:0/2416984211 conn(0x7fa9e0073660 msgr2=0x7fa9e01596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 shutdown_connections 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7fa9e5a39640 1 -- 172.21.15.105:0/2416984211 wait complete. 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f2fdc082380 msgr2=0x7f2fdc084810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f2fdc082380 0x7f2fdc084810 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2fdc078bc0 msgr2=0x7f2fdc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f2fdc078bc0 0x7f2fdc07b0d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0001400 tx=0x7f2fe0016040 comp rx=0 tx=0).stop 2024-09-21T07:26:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 msgr2=0x7f2ff014fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 --2- 172.21.15.105:0/161775840 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ff00ce1f0 0x7f2ff014fcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009bf0 tx=0x7f2fe80045e0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f21fffff640 1 -- 172.21.15.105:0/4066364569 --> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21f4002d50 con 0x7f21f8082350 2024-09-21T07:26:38.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2feffff640 1 -- 172.21.15.105:0/161775840 reap_dead start 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 shutdown_connections 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 >> 172.21.15.105:0/161775840 conn(0x7f2ff00ccb90 msgr2=0x7f2ff00aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 shutdown_connections 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.809+0000 7f2ff5dad640 1 -- 172.21.15.105:0/161775840 wait complete. 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.810+0000 7f220a7fc640 1 -- 172.21.15.105:0/4066364569 <== osd.5 v2:172.21.15.112:6826/2151662407 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21f4002d50 con 0x7f21f8082350 2024-09-21T07:26:38.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.810+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 --> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fddd0002dd0 con 0x7fddcc082460 2024-09-21T07:26:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.810+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f21f8082350 msgr2=0x7f21f80847e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.810+0000 7f221aae1640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f21f8082350 0x7f21f80847e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.811+0000 7fdde2ffd640 1 -- 172.21.15.105:0/2234476160 <== osd.2 v2:172.21.15.105:6808/3103137694 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fddd0002dd0 con 0x7fddcc082460 2024-09-21T07:26:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7fddcc082460 msgr2=0x7fddcc0848f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7fdde0ff9640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7fddcc082460 0x7fddcc0848f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f21f8078c10 msgr2=0x7f21f807b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7f221aae1640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f21f8078c10 0x7f21f807b120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2210034000 tx=0x7f2210004730 comp rx=0 tx=0).stop 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c0e1cc0 msgr2=0x7f220c0e20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7f221aae1640 1 --2- 172.21.15.105:0/4066364569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f220c0e1cc0 0x7f220c0e20d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f221406a320 tx=0x7f221406a0e0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.812+0000 7f221a2e0640 1 -- 172.21.15.105:0/4066364569 reap_dead start 2024-09-21T07:26:38.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 shutdown_connections 2024-09-21T07:26:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 >> 172.21.15.105:0/4066364569 conn(0x7f220c051420 msgr2=0x7f220c053c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 --> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f816c0d8040 con 0x7f8158082430 2024-09-21T07:26:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 shutdown_connections 2024-09-21T07:26:38.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 --> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f68002d50 con 0x7f1f64082430 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f221aae1640 1 -- 172.21.15.105:0/4066364569 wait complete. 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fddcc078c90 msgr2=0x7fddcc07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fddcc078c90 0x7fddcc07b1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdde400abe0 tx=0x7fdde4005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec140ff0 msgr2=0x7fddec0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 --2- 172.21.15.105:0/2234476160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fddec140ff0 0x7fddec0b1d50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fddd400c390 tx=0x7fddd400c850 comp rx=0 tx=0).stop 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fddead9d640 1 -- 172.21.15.105:0/2234476160 reap_dead start 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 shutdown_connections 2024-09-21T07:26:38.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 >> 172.21.15.105:0/2234476160 conn(0x7fddec06cda0 msgr2=0x7fddec0b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f8169ffb640 1 -- 172.21.15.105:0/3191902962 <== osd.7 v2:172.21.15.112:6802/3904329758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f816c0d8040 con 0x7f8158082430 2024-09-21T07:26:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 shutdown_connections 2024-09-21T07:26:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7fdde0ff9640 1 -- 172.21.15.105:0/2234476160 wait complete. 2024-09-21T07:26:38.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.813+0000 7f1f76ffd640 1 -- 172.21.15.105:0/336293575 <== osd.1 v2:172.21.15.105:6816/4277636459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f68002d50 con 0x7f1f64082430 2024-09-21T07:26:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f8158082430 msgr2=0x7f81580848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f817a561640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f8158082430 0x7f81580848c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f1f64082430 msgr2=0x7f1f640848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f1f87440640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f1f64082430 0x7f1f640848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8158078c60 msgr2=0x7f815807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f817a561640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8158078c60 0x7f815807b170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f81700058c0 tx=0x7f8170009260 comp rx=0 tx=0).stop 2024-09-21T07:26:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.814+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 msgr2=0x7f816c0d75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f817a561640 1 --2- 172.21.15.105:0/3191902962 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f816c054190 0x7f816c0d75d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f816400d6b0 tx=0x7f816400dab0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f8179d60640 1 -- 172.21.15.105:0/3191902962 reap_dead start 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1f64078c60 msgr2=0x7f1f6407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1f64078c60 0x7f1f6407b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1f780dfcb0 tx=0x7f1f70007970 comp rx=0 tx=0).stop 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 msgr2=0x7f1f7817e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 --2- 172.21.15.105:0/336293575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f78054b70 0x7f1f7817e660 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0078f0 tx=0x7f1f7c007db0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f86c3f640 1 -- 172.21.15.105:0/336293575 reap_dead start 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 shutdown_connections 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 >> 172.21.15.105:0/336293575 conn(0x7f1f7801ddf0 msgr2=0x7f1f7800a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 shutdown_connections 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 shutdown_connections 2024-09-21T07:26:38.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 >> 172.21.15.105:0/3191902962 conn(0x7f816c03a680 msgr2=0x7f816c04c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f1f87440640 1 -- 172.21.15.105:0/336293575 wait complete. 2024-09-21T07:26:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 shutdown_connections 2024-09-21T07:26:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.815+0000 7f817a561640 1 -- 172.21.15.105:0/3191902962 wait complete. 2024-09-21T07:26:38.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.817+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 --> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7678181d10 con 0x7f7664082430 2024-09-21T07:26:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.817+0000 7f767cff9640 1 -- 172.21.15.105:0/1654436320 <== osd.6 v2:172.21.15.112:6818/2596982695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7678181d10 con 0x7f7664082430 2024-09-21T07:26:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f7664082430 msgr2=0x7f76640848c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f7664082430 0x7f76640848c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7664078c60 msgr2=0x7f766407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7664078c60 0x7f766407b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7678054950 tx=0x7f7674005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 msgr2=0x7f7678053aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.818+0000 7f76857c3640 1 --2- 172.21.15.105:0/1654436320 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7678006050 0x7f7678053aa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bad0 tx=0x7f766c035ba0 comp rx=0 tx=0).stop 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.819+0000 7f7684fc2640 1 -- 172.21.15.105:0/1654436320 reap_dead start 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.819+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 shutdown_connections 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.819+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 >> 172.21.15.105:0/1654436320 conn(0x7f767801ddf0 msgr2=0x7f7678036660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.820+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 shutdown_connections 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:38.820+0000 7f76857c3640 1 -- 172.21.15.105:0/1654436320 wait complete. 2024-09-21T07:26:38.848 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2024-09-21T07:26:38.849 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-09-21T07:26:38.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:26:38.850 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:26:38.851 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:38.851 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:26:38.852 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-09-21T07:26:38.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:26:38.853 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-09-21T07:26:38.853 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:26:38.854 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:38.854 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:26:38.855 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-09-21T07:26:38.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:26:38.856 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:38.856 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:26:39.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.101+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/3701390211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a8073df0 msgr2=0x7fa9a8074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.101+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/3701390211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a8073df0 0x7fa9a8074200 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa9a80d1930 tx=0x7fa99802a450 comp rx=0 tx=0).stop 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/3701390211 shutdown_connections 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/3701390211 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a8073df0 0x7fa9a8074200 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/3701390211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a815e710 0x7fa9a815eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/3701390211 >> v2:172.21.15.112:3300/0 conn(0x7fa9a8151d80 0x7fa9a815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/3701390211 >> 172.21.15.105:0/3701390211 conn(0x7fa9a80d1340 msgr2=0x7fa9a80736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/3701390211 shutdown_connections 2024-09-21T07:26:39.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.102+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/3701390211 wait complete. 2024-09-21T07:26:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.105+0000 7f030395f640 1 -- 172.21.15.105:0/2782759020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f40519a0 msgr2=0x7f02f4051db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.105+0000 7f030395f640 1 --2- 172.21.15.105:0/2782759020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f40519a0 0x7f02f4051db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f02fc06b230 tx=0x7f02fc0a6720 comp rx=0 tx=0).stop 2024-09-21T07:26:39.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.105+0000 7fa9ad4fd640 1 Processor -- start 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 -- 172.21.15.105:0/2782759020 shutdown_connections 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 --2- 172.21.15.105:0/2782759020 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f40519a0 0x7f02f4051db0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 --2- 172.21.15.105:0/2782759020 >> v2:172.21.15.105:3300/0 conn(0x7f02f40df800 0x7f02f4007ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 -- 172.21.15.105:0/2782759020 >> 172.21.15.105:0/2782759020 conn(0x7f02f401e260 msgr2=0x7f02f4039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 -- 172.21.15.105:0/3164488672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa4052360 msgr2=0x7f7aa40e7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/3164488672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa4052360 0x7f7aa40e7280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4055860 tx=0x7f7aa802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 -- 172.21.15.105:0/2782759020 shutdown_connections 2024-09-21T07:26:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fa9ad4fd640 1 -- start start 2024-09-21T07:26:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f030395f640 1 -- 172.21.15.105:0/2782759020 wait complete. 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 -- 172.21.15.105:0/2736842284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc100cefc0 msgr2=0x7fdc10073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/2736842284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc100cefc0 0x7fdc10073370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdc100c3b70 tx=0x7fdc0802a730 comp rx=0 tx=0).stop 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 -- 172.21.15.105:0/2736842284 shutdown_connections 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/2736842284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc100cefc0 0x7fdc10073370 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/2736842284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc10071010 0x7fdc100ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 -- 172.21.15.105:0/2736842284 >> 172.21.15.105:0/2736842284 conn(0x7fdc100ccb90 msgr2=0x7fdc100ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 -- 172.21.15.105:0/2736842284 shutdown_connections 2024-09-21T07:26:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 -- 172.21.15.105:0/3164488672 shutdown_connections 2024-09-21T07:26:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/3164488672 >> v2:172.21.15.112:3300/0 conn(0x7f7aa4054720 0x7f7aa4054b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/3164488672 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa4052360 0x7f7aa40e7280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7f7ab22d7640 1 -- 172.21.15.105:0/3164488672 >> 172.21.15.105:0/3164488672 conn(0x7f7aa401d9a0 msgr2=0x7f7aa4039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.106+0000 7fdc15d5e640 1 -- 172.21.15.105:0/2736842284 wait complete. 2024-09-21T07:26:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.107+0000 7f7ab22d7640 1 -- 172.21.15.105:0/3164488672 shutdown_connections 2024-09-21T07:26:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.107+0000 7f7ab22d7640 1 -- 172.21.15.105:0/3164488672 wait complete. 2024-09-21T07:26:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.107+0000 7fcf4a7fc640 1 -- 172.21.15.105:0/2632824760 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44036400 con 0x7fcf4c0710f0 2024-09-21T07:26:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fa9ad4fd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 -- 172.21.15.105:0/2632824760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0710f0 msgr2=0x7fcf4c070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 --2- 172.21.15.105:0/2632824760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c070090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005fd0 tx=0x7fcf4402aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 -- 172.21.15.105:0/2632824760 shutdown_connections 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 --2- 172.21.15.105:0/2632824760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c070090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 --2- 172.21.15.105:0/2632824760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf4c15f970 0x7fcf4c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.108+0000 7fcf50bce640 1 -- 172.21.15.105:0/2632824760 >> 172.21.15.105:0/2632824760 conn(0x7fcf4c0b1c80 msgr2=0x7fcf4c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7ff314f72640 1 -- 172.21.15.105:0/1877738525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 msgr2=0x7ff2f800ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7ff314f72640 1 --2- 172.21.15.105:0/1877738525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff2f800ba00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff30800c560 tx=0x7ff2f8005830 comp rx=0 tx=0).stop 2024-09-21T07:26:39.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9ad4fd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a8151d80 0x7fa9a81fa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fcf50bce640 1 -- 172.21.15.105:0/2632824760 shutdown_connections 2024-09-21T07:26:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9ad4fd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 0x7fa9a814b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9a7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38400/0 (socket says 172.21.15.105:38400) 2024-09-21T07:26:39.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fcf50bce640 1 -- 172.21.15.105:0/2632824760 wait complete. 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9a7fff640 1 -- 172.21.15.105:0/660152720 learned_addr learned my addr 172.21.15.105:0/660152720 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a8069840 con 0x7fa9a815e710 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9accfc640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 0x7fa9a814b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a80696c0 con 0x7fa9a8151d80 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a80699c0 con 0x7fa9a8073df0 2024-09-21T07:26:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7f030395f640 1 Processor -- start 2024-09-21T07:26:39.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7fa9a77fe640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a8151d80 0x7fa9a81fa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7ff314f72640 1 -- 172.21.15.105:0/1877738525 shutdown_connections 2024-09-21T07:26:39.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7f030395f640 1 -- start start 2024-09-21T07:26:39.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7ff314f72640 1 --2- 172.21.15.105:0/1877738525 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff2f800ba00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.109+0000 7ff314f72640 1 -- 172.21.15.105:0/1877738525 >> 172.21.15.105:0/1877738525 conn(0x7ff3100d0910 msgr2=0x7ff3100d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 -- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a8151d80 msgr2=0x7fa9a81fa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a8151d80 0x7fa9a81fa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 -- 172.21.15.105:0/660152720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 msgr2=0x7fa9a814b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 0x7fa9a814b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a81fa560 con 0x7fa9a8073df0 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9accfc640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 0x7fa9a814b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:39.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fdc15d5e640 1 Processor -- start 2024-09-21T07:26:39.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7ff314f72640 1 -- 172.21.15.105:0/1877738525 shutdown_connections 2024-09-21T07:26:39.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7ff314f72640 1 -- 172.21.15.105:0/1877738525 wait complete. 2024-09-21T07:26:39.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a7fff640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000ba80 tx=0x7fa9a000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fdc15d5e640 1 -- start start 2024-09-21T07:26:39.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0019070 con 0x7fa9a8073df0 2024-09-21T07:26:39.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.110+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9a000cd10 con 0x7fa9a8073df0 2024-09-21T07:26:39.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.111+0000 7f7ab22d7640 1 Processor -- start 2024-09-21T07:26:39.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.111+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00146a0 con 0x7fa9a8073df0 2024-09-21T07:26:39.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f030395f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fc09f0d3640 1 -- 172.21.15.105:0/1858046209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0900dfe70 msgr2=0x7fc090008d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/1858046209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0900dfe70 0x7fc090008d90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009820 tx=0x7fc08c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a814b9d0 con 0x7fa9a8073df0 2024-09-21T07:26:39.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f030395f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 0x7f02f404e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fdc15d5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc10071010 0x7fdc1009cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.111+0000 7f7ab22d7640 1 -- start start 2024-09-21T07:26:39.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f7ab22d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa4051630 0x7f7aa40314c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f7ab22d7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 0x7f7aa404cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a80ce630 con 0x7fa9a8073df0 2024-09-21T07:26:39.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f7ab22d7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 0x7f7aa404d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fdc15d5e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 0x7fdc1009b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7f7ab22d7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4052770 con 0x7f7aa40d44a0 2024-09-21T07:26:39.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab22d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa4033ec0 con 0x7f7aa4051630 2024-09-21T07:26:39.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab22d7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa40528f0 con 0x7f7aa4054720 2024-09-21T07:26:39.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc15d5e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 0x7fdc1009bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab1ad6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 0x7f7aa404d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc15d5e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10073e30 con 0x7fdc1009bb80 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 -- 172.21.15.105:0/1858046209 shutdown_connections 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/1858046209 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc09003af10 0x7fc0900081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/1858046209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0900dfe70 0x7fc090008d90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 -- 172.21.15.105:0/1858046209 >> 172.21.15.105:0/1858046209 conn(0x7fc09001eb10 msgr2=0x7fc09003a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 -- 172.21.15.105:0/1858046209 shutdown_connections 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fc09f0d3640 1 -- 172.21.15.105:0/1858046209 wait complete. 2024-09-21T07:26:39.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab0ad4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 0x7f7aa404cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 0x7fdc1009b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f03016fb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f03016fb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38408/0 (socket says 172.21.15.105:38408) 2024-09-21T07:26:39.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc06ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 0x7fdc1009b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38430/0 (socket says 172.21.15.105:38430) 2024-09-21T07:26:39.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc06ffd640 1 -- 172.21.15.105:0/521317580 learned_addr learned my addr 172.21.15.105:0/521317580 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdc10073cb0 con 0x7fdc10071010 2024-09-21T07:26:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fdc10073fb0 con 0x7fdc100cb370 2024-09-21T07:26:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7ff314f72640 1 Processor -- start 2024-09-21T07:26:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f03016fb640 1 -- 172.21.15.105:0/3455296160 learned_addr learned my addr 172.21.15.105:0/3455296160 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab1ad6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 0x7f7aa404d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59954/0 (socket says 172.21.15.105:59954) 2024-09-21T07:26:39.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f40df800 0x7f02f404e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7fdc0ffff640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 0x7fdc1009bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7ff314f72640 1 -- start start 2024-09-21T07:26:39.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4054720 con 0x7f02f40519a0 2024-09-21T07:26:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02f40548a0 con 0x7f02f40df800 2024-09-21T07:26:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f02f4054a20 con 0x7f02f4006860 2024-09-21T07:26:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f7ab1ad6640 1 -- 172.21.15.105:0/2218435658 learned_addr learned my addr 172.21.15.105:0/2218435658 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f0301efc640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f40df800 0x7f02f404e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f0300efa640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 0x7f02f404e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fcf50bce640 1 Processor -- start 2024-09-21T07:26:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab12d5640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa4051630 0x7f7aa40314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.112+0000 7fcf50bce640 1 -- start start 2024-09-21T07:26:39.130 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fdc0ffff640 1 -- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 msgr2=0x7fdc1009b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fdc0ffff640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 0x7fdc1009b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fdc0ffff640 1 -- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc10071010 msgr2=0x7fdc1009cb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 -- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa4051630 msgr2=0x7f7aa40314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa4051630 0x7f7aa40314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 -- 172.21.15.105:0/2218435658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 msgr2=0x7f7aa404d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 0x7f7aa404d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 -- 172.21.15.105:0/2218435658 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa404e060 con 0x7f7aa4054720 2024-09-21T07:26:39.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.113+0000 7f72b4be4640 1 -- 172.21.15.105:0/3368008883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 msgr2=0x7f729800bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 --2- 172.21.15.105:0/3368008883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f729800bbc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f72a8002100 tx=0x7f7298007a40 comp rx=0 tx=0).stop 2024-09-21T07:26:39.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fdc0ffff640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc10071010 0x7fdc1009cb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fdc0ffff640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc101510e0 con 0x7fdc1009bb80 2024-09-21T07:26:39.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f7ab0ad4640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 0x7f7aa404cfb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b410 tx=0x7f7a9c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fcf50bce640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 -- 172.21.15.105:0/3368008883 shutdown_connections 2024-09-21T07:26:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 --2- 172.21.15.105:0/3368008883 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f729800bbc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 -- 172.21.15.105:0/3368008883 >> 172.21.15.105:0/3368008883 conn(0x7f72b00ccb90 msgr2=0x7f72b00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fcf50bce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 -- 172.21.15.105:0/3368008883 shutdown_connections 2024-09-21T07:26:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7fcf50bce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf4c15f970 0x7fcf4c0cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.114+0000 7f72b4be4640 1 -- 172.21.15.105:0/3368008883 wait complete. 2024-09-21T07:26:39.134 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c015020 con 0x7f7aa4054720 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf50bce640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c162bb0 con 0x7fcf4c0705d0 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fdc0ffff640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 0x7fdc1009bf90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009000 tx=0x7fdc08032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08041020 con 0x7fdc1009bb80 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1014b920 con 0x7fdc1009bb80 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1014bee0 con 0x7fdc1009bb80 2024-09-21T07:26:39.135 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a815bba0 con 0x7fa9a8073df0 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9a001c070 con 0x7fa9a8073df0 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a9c011920 con 0x7f7aa4054720 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff314f72640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff3101e1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff314f72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59968/0 (socket says 172.21.15.105:59968) 2024-09-21T07:26:39.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4b7fe640 1 -- 172.21.15.105:0/1661344410 learned_addr learned my addr 172.21.15.105:0/1661344410 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c162a30 con 0x7fcf4c0710f0 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c162d30 con 0x7fcf4c15f970 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4affd640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4b7fe640 1 -- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf4c15f970 msgr2=0x7fcf4c0cdb20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fcf4b7fe640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf4c15f970 0x7fcf4c0cdb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf4b7fe640 1 -- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 msgr2=0x7fcf4c148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf4b7fe640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf4b7fe640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1f67a0 con 0x7fcf4c0705d0 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf4affd640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf4b7fe640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e790 tx=0x7fcf3c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff3101e1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff3101e1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38444/0 (socket says 172.21.15.105:38444) 2024-09-21T07:26:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fc09f0d3640 1 Processor -- start 2024-09-21T07:26:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fa9a4ff9640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 0x7fa99407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa404a330 con 0x7f7aa4054720 2024-09-21T07:26:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff30dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:59978/0 (socket says 172.21.15.105:59978) 2024-09-21T07:26:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fc09f0d3640 1 -- start start 2024-09-21T07:26:39.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff30e59c640 1 -- 172.21.15.105:0/4061781078 learned_addr learned my addr 172.21.15.105:0/4061781078 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fa9a77fe640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 0x7fa99407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 0x7ff3101e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa404e5e0 con 0x7f7aa4054720 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3100b3ce0 con 0x7ff3101459b0 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3100b3b60 con 0x7ff3101e2ae0 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff3100b3e60 con 0x7ff2f8005350 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c011db0 con 0x7f7aa4054720 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.115+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc08037040 con 0x7fdc1009bb80 2024-09-21T07:26:39.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc08041020 con 0x7fdc1009bb80 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7ff30ed9d640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 0x7ff3101e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa9a009adf0 con 0x7fa9a8073df0 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7f03016fb640 1 -- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f40df800 msgr2=0x7f02f404e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7f03016fb640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f40df800 0x7f02f404e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7f03016fb640 1 -- 172.21.15.105:0/3455296160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 msgr2=0x7f02f404e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7f03016fb640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 0x7f02f404e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7f03016fb640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f401fcc0 con 0x7f02f4006860 2024-09-21T07:26:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7fa9a77fe640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 0x7fa99407b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00a490 tx=0x7fa99c00a420 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7f0300efa640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 0x7f02f404e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00eea0 con 0x7fcf4c0705d0 2024-09-21T07:26:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf3c00cdb0 con 0x7fcf4c0705d0 2024-09-21T07:26:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c010690 con 0x7fcf4c0705d0 2024-09-21T07:26:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.116+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c0cd040 con 0x7fcf4c0705d0 2024-09-21T07:26:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.117+0000 7f03016fb640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b440 tx=0x7f02f800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1f63d0 con 0x7fcf4c0705d0 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 -- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 msgr2=0x7ff3101e1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff3101e1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 -- 172.21.15.105:0/4061781078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 msgr2=0x7ff3101e25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3101e3430 con 0x7ff3101e2ae0 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30ed9d640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 0x7ff3101e2ef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff31006f6e0 tx=0x7ff308004bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7f72b4be4640 1 Processor -- start 2024-09-21T07:26:39.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff30dd9b640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:39.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7f72b4be4640 1 -- start start 2024-09-21T07:26:39.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30802bc30 con 0x7ff3101e2ae0 2024-09-21T07:26:39.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff30802bdd0 con 0x7ff3101e2ae0 2024-09-21T07:26:39.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf4005290 con 0x7fdc1009bb80 2024-09-21T07:26:39.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30802bc30 con 0x7ff3101e2ae0 2024-09-21T07:26:39.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3101557f0 con 0x7ff3101e2ae0 2024-09-21T07:26:39.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.118+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff310155db0 con 0x7ff3101e2ae0 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09003af10 0x7fc0900d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa40552f0 con 0x7f7aa4054720 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 0x7fc0900d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09ce6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc090053580 con 0x7fc0900dfe70 2024-09-21T07:26:39.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc090053400 con 0x7fc09003af10 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09ce6f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38454/0 (socket says 172.21.15.105:38454) 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09ce6f640 1 -- 172.21.15.105:0/296114550 learned_addr learned my addr 172.21.15.105:0/296114550 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c070090 con 0x7fcf4c0705d0 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09f0d3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc090053700 con 0x7fc090008730 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc09d670640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 0x7fc0900d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.119+0000 7fc097fff640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09003af10 0x7fc0900d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 -- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09003af10 msgr2=0x7fc0900d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09003af10 0x7fc0900d35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 -- 172.21.15.105:0/296114550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 msgr2=0x7fc0900d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 0x7fc0900d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 -- 172.21.15.105:0/296114550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc090050f40 con 0x7fc090008730 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09d670640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 0x7fc0900d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3100af900 con 0x7ff3101e2ae0 2024-09-21T07:26:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7fc09ce6f640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009770 tx=0x7fc08c0055e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72b00bed60 0x7f72b00c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b009d560 0x7f72b00c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60000/0 (socket says 172.21.15.105:60000) 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00cb980 con 0x7f72980073b0 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72b00cb800 con 0x7f72b009d560 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72b4be4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f72b00cbb00 con 0x7f72b00bed60 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72ae59c640 1 -- 172.21.15.105:0/370500379 learned_addr learned my addr 172.21.15.105:0/370500379 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72add9b640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72b00bed60 0x7f72b00c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.120+0000 7f72aed9d640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b009d560 0x7f72b00c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf3c010830 con 0x7fcf4c0705d0 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 -- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72b00bed60 msgr2=0x7f72b00c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72b00bed60 0x7f72b00c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 -- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b009d560 msgr2=0x7f72b00c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b009d560 0x7f72b00c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 -- 172.21.15.105:0/370500379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b00b4fa0 con 0x7f72980073b0 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7f72ae59c640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72a00068f0 tx=0x7f72a00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7fcf33fff640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 0x7fcf3807b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc08044020 con 0x7fdc1009bb80 2024-09-21T07:26:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff308006030 con 0x7ff3101e2ae0 2024-09-21T07:26:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.121+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcf3c010430 con 0x7fcf4c0705d0 2024-09-21T07:26:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9a814b9d0 con 0x7fa9a8073df0 2024-09-21T07:26:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fdc0cff9640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 0x7fdbf007b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c033e40 con 0x7fc090008730 2024-09-21T07:26:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.123+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdc08031440 con 0x7fdc1009bb80 2024-09-21T07:26:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7ff2feffd640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 0x7ff2f007b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff30802ba30 con 0x7ff3101e2ae0 2024-09-21T07:26:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8015020 con 0x7f02f4006860 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f40216f0 con 0x7f02f4006860 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f8011980 con 0x7f02f4006860 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8015020 con 0x7f02f4006860 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f4021cb0 con 0x7f02f4006860 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc08c004260 con 0x7fc090008730 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0900511e0 con 0x7fc090008730 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0900517a0 con 0x7fc090008730 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c031950 con 0x7fc090008730 2024-09-21T07:26:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.123+0000 7fdc0f7fe640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 0x7fdbf007b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fcf4affd640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 0x7fcf3807b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.122+0000 7fcf4affd640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 0x7fcf3807b2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf44035000 tx=0x7fcf44004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.123+0000 7ff30e59c640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 0x7ff2f007b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.157 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7fdc0f7fe640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 0x7fdbf007b4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc000099c0 tx=0x7fdc00005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7ff30e59c640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 0x7ff2f007b4a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff300001760 tx=0x7ff300009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7a9c018020 con 0x7f7aa4054720 2024-09-21T07:26:39.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078005290 con 0x7fc090008730 2024-09-21T07:26:39.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02dc005290 con 0x7f02f4006860 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.124+0000 7f7aa1ffb640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 0x7f7a9007b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.125+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7a9c09b450 con 0x7f7aa4054720 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.125+0000 7f7ab12d5640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 0x7f7a9007b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.125+0000 7f7ab12d5640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 0x7f7a9007b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8034000 tx=0x7f7aa8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.128+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a9c01f020 con 0x7f7aa4054720 2024-09-21T07:26:39.159 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.128+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff308044030 con 0x7ff3101e2ae0 2024-09-21T07:26:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.129+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc08083950 con 0x7fdc1009bb80 2024-09-21T07:26:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.130+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f02f8018020 con 0x7f02f4006860 2024-09-21T07:26:39.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.129+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc08c031af0 con 0x7fc090008730 2024-09-21T07:26:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.130+0000 7fc0957fa640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 0x7fc07c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.130+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc08c0bfcf0 con 0x7fc090008730 2024-09-21T07:26:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.131+0000 7fc097fff640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 0x7fc07c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.131+0000 7f02f1ffb640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 0x7f02e007b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.161 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.131+0000 7fc097fff640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 0x7fc07c07b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0900df2c0 tx=0x7fc084005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.131+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f02f809afa0 con 0x7f02f4006860 2024-09-21T07:26:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.131+0000 7f0300efa640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 0x7f02e007b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.132+0000 7f0300efa640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 0x7f02e007b2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02f40377a0 tx=0x7f02fc06aa90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.133+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc08c04c030 con 0x7fc090008730 2024-09-21T07:26:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf3c063950 con 0x7fcf4c0705d0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.138+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0015070 con 0x7f72980073b0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a0011900 con 0x7f72980073b0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0011d50 con 0x7f72980073b0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b00b57e0 con 0x7f72980073b0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b00b5a80 con 0x7f72980073b0 2024-09-21T07:26:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.139+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02f801f020 con 0x7f02f4006860 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.140+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0070c50 con 0x7f72980073b0 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.144+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f72a0018070 con 0x7f72980073b0 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.145+0000 7f729effd640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 0x7f729007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.145+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f72a009bd50 con 0x7f72980073b0 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.146+0000 7f72add9b640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 0x7f729007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.156+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f72b0070c50 con 0x7f72980073b0 2024-09-21T07:26:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.156+0000 7f72add9b640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 0x7f729007b170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72a400f780 tx=0x7f72a4011040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.549+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc078005850 con 0x7fc090008730 2024-09-21T07:26:39.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.549+0000 7fc0957fa640 1 -- 172.21.15.105:0/296114550 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc08c087d50 con 0x7fc090008730 2024-09-21T07:26:39.551 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-09-21T07:26:39.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.552+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7aa4033710 con 0x7f7aa4054720 2024-09-21T07:26:39.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.552+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa9a80cead0 con 0x7fa9a8073df0 2024-09-21T07:26:39.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.552+0000 7f7aa1ffb640 1 -- 172.21.15.105:0/2218435658 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7a9c063630 con 0x7f7aa4054720 2024-09-21T07:26:39.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.553+0000 7fa9a4ff9640 1 -- 172.21.15.105:0/660152720 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9a81fa560 con 0x7fa9a8073df0 2024-09-21T07:26:39.555 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-09-21T07:26:39.555 INFO:teuthology.orchestra.run.smithi105.stdout:55834574857 2024-09-21T07:26:39.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.553+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 msgr2=0x7fc07c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.553+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 0x7fc07c07b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0900df2c0 tx=0x7fc084005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.553+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 msgr2=0x7fc090176de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.553+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009770 tx=0x7fc08c0055e0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 shutdown_connections 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc07c078bc0 0x7fc07c07b0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc09003af10 0x7fc0900d35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc0900dfe70 0x7fc0900d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 --2- 172.21.15.105:0/296114550 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc090008730 0x7fc090176de0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 >> 172.21.15.105:0/296114550 conn(0x7fc09001eb10 msgr2=0x7fc090053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 shutdown_connections 2024-09-21T07:26:39.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.554+0000 7fc09f0d3640 1 -- 172.21.15.105:0/296114550 wait complete. 2024-09-21T07:26:39.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 msgr2=0x7f7a9007b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 msgr2=0x7fa99407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 0x7fa99407b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00a490 tx=0x7fa99c00a420 comp rx=0 tx=0).stop 2024-09-21T07:26:39.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 msgr2=0x7fa9a8152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000ba80 tx=0x7fa9a000bf40 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 0x7f7a9007b0f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8034000 tx=0x7f7aa8004730 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 msgr2=0x7f7aa404cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 0x7f7aa404cfb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00b410 tx=0x7f7a9c00b810 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 shutdown_connections 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7a90078be0 0x7f7a9007b0f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aa40d44a0 0x7f7aa404d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7aa4054720 0x7f7aa404cfb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 --2- 172.21.15.105:0/2218435658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7aa4051630 0x7f7aa40314c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 >> 172.21.15.105:0/2218435658 conn(0x7f7aa401d9a0 msgr2=0x7f7aa4004bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 shutdown_connections 2024-09-21T07:26:39.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.556+0000 7f7ab22d7640 1 -- 172.21.15.105:0/2218435658 wait complete. 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 shutdown_connections 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa994078c90 0x7fa99407b1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9a8151d80 0x7fa9a81fa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9a815e710 0x7fa9a814b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 --2- 172.21.15.105:0/660152720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa9a8073df0 0x7fa9a8152240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 >> 172.21.15.105:0/660152720 conn(0x7fa9a80d1340 msgr2=0x7fa9a815db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 shutdown_connections 2024-09-21T07:26:39.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.557+0000 7fa9ad4fd640 1 -- 172.21.15.105:0/660152720 wait complete. 2024-09-21T07:26:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.558+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff3101e21c0 con 0x7ff3101e2ae0 2024-09-21T07:26:39.563 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.559+0000 7ff2feffd640 1 -- 172.21.15.105:0/4061781078 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff308044030 con 0x7ff3101e2ae0 2024-09-21T07:26:39.564 INFO:teuthology.orchestra.run.smithi105.stdout:64424509448 2024-09-21T07:26:39.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.560+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdbf4005850 con 0x7fdc1009bb80 2024-09-21T07:26:39.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.561+0000 7fdc0cff9640 1 -- 172.21.15.105:0/521317580 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc08087d30 con 0x7fdc1009bb80 2024-09-21T07:26:39.565 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.563+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 msgr2=0x7ff2f007b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.563+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 0x7ff2f007b4a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff300001760 tx=0x7ff300009040 comp rx=0 tx=0).stop 2024-09-21T07:26:39.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 msgr2=0x7ff3101e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 0x7ff3101e2ef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff31006f6e0 tx=0x7ff308004bb0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 shutdown_connections 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff2f0078f90 0x7ff2f007b4a0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff300001760 tx=0x7ff300009040 comp rx=0 tx=0).stop 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3101e2ae0 0x7ff3101e2ef0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3101459b0 0x7ff3101e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 --2- 172.21.15.105:0/4061781078 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2f8005350 0x7ff3101e1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 >> 172.21.15.105:0/4061781078 conn(0x7ff3100d0910 msgr2=0x7ff310157330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.564+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 shutdown_connections 2024-09-21T07:26:39.567 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7ff314f72640 1 -- 172.21.15.105:0/4061781078 wait complete. 2024-09-21T07:26:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 msgr2=0x7fdbf007b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 0x7fdbf007b4a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc000099c0 tx=0x7fdc00005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 msgr2=0x7fdc1009bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 0x7fdc1009bf90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009000 tx=0x7fdc08032de0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 shutdown_connections 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fdbf0078f90 0x7fdbf007b4a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdc1009bb80 0x7fdc1009bf90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc100cb370 0x7fdc1009b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 --2- 172.21.15.105:0/521317580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc10071010 0x7fdc1009cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.565+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 >> 172.21.15.105:0/521317580 conn(0x7fdc100ccb90 msgr2=0x7fdc100b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.567+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 shutdown_connections 2024-09-21T07:26:39.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.567+0000 7fdc15d5e640 1 -- 172.21.15.105:0/521317580 wait complete. 2024-09-21T07:26:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.568+0000 7fcf50bce640 1 -- 172.21.15.105:0/1661344410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcf4c15f370 con 0x7fcf4c0705d0 2024-09-21T07:26:39.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.569+0000 7fcf33fff640 1 -- 172.21.15.105:0/1661344410 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf3c063950 con 0x7fcf4c0705d0 2024-09-21T07:26:39.571 INFO:teuthology.orchestra.run.smithi105.stdout:64424509449 2024-09-21T07:26:39.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.572+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f02dc005850 con 0x7f02f4006860 2024-09-21T07:26:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 msgr2=0x7fcf3807b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 0x7fcf3807b2a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf44035000 tx=0x7fcf44004ab0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 msgr2=0x7fcf4c156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e790 tx=0x7fcf3c00ec50 comp rx=0 tx=0).stop 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 shutdown_connections 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcf38078d90 0x7fcf3807b2a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcf4c0710f0 0x7fcf4c148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcf4c15f970 0x7fcf4c0cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 --2- 172.21.15.105:0/1661344410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf4c0705d0 0x7fcf4c156480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.573+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 >> 172.21.15.105:0/1661344410 conn(0x7fcf4c0b1c80 msgr2=0x7fcf4c161780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.574+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 shutdown_connections 2024-09-21T07:26:39.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.574+0000 7fcf31ffb640 1 -- 172.21.15.105:0/1661344410 wait complete. 2024-09-21T07:26:39.577 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.574+0000 7f02f1ffb640 1 -- 172.21.15.105:0/3455296160 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f02f800d040 con 0x7f02f4006860 2024-09-21T07:26:39.577 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:39.579 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.577+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f72b006f1f0 con 0x7f72980073b0 2024-09-21T07:26:39.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-21T07:26:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.578+0000 7f729effd640 1 -- 172.21.15.105:0/370500379 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f72a0063ed0 con 0x7f72980073b0 2024-09-21T07:26:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.578+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 msgr2=0x7f02e007b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.580 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.578+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 0x7f02e007b2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f02f40377a0 tx=0x7f02fc06aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:39.580 INFO:teuthology.orchestra.run.smithi105.stdout:64424509448 2024-09-21T07:26:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.578+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 msgr2=0x7f02f40d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.578+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02f800b440 tx=0x7f02f800b900 comp rx=0 tx=0).stop 2024-09-21T07:26:39.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.5 2024-09-21T07:26:39.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2024-09-21T07:26:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 shutdown_connections 2024-09-21T07:26:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f02e0078d90 0x7f02e007b2a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02f40df800 0x7f02f404e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f02f40519a0 0x7f02f404e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 --2- 172.21.15.105:0/3455296160 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02f4006860 0x7f02f40d3060 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 >> 172.21.15.105:0/3455296160 conn(0x7f02f401e260 msgr2=0x7f02f4005600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 shutdown_connections 2024-09-21T07:26:39.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.579+0000 7f030395f640 1 -- 172.21.15.105:0/3455296160 wait complete. 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 msgr2=0x7f729007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 0x7f729007b170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f72a400f780 tx=0x7f72a4011040 comp rx=0 tx=0).stop 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 msgr2=0x7f72b0157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f72a00068f0 tx=0x7f72a00039c0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 shutdown_connections 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7290078c60 0x7f729007b170 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72b009d560 0x7f72b00c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f72b00bed60 0x7f72b00c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 --2- 172.21.15.105:0/370500379 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72980073b0 0x7f72b0157600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 >> 172.21.15.105:0/370500379 conn(0x7f72b00ccb90 msgr2=0x7f72b00acf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 shutdown_connections 2024-09-21T07:26:39.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:39.582+0000 7f72b4be4640 1 -- 172.21.15.105:0/370500379 wait complete. 2024-09-21T07:26:39.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.0 2024-09-21T07:26:39.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.4 2024-09-21T07:26:39.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.2 2024-09-21T07:26:39.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-21T07:26:39.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.1 2024-09-21T07:26:40.580 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:26:40.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:26:40.583 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:26:40.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:26:40.592 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:26:40.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:26:40.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:26:40.599 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:26:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7f684d8fa640 1 -- 172.21.15.105:0/3581473972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6848071010 msgr2=0x7f68480ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7f684d8fa640 1 --2- 172.21.15.105:0/3581473972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6848071010 0x7f68480ca760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6848076450 tx=0x7f68380329d0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 -- 172.21.15.105:0/1971344553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 msgr2=0x7fa8840d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1971344553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840d0a50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005fd0 tx=0x7fa87c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 -- 172.21.15.105:0/1971344553 shutdown_connections 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1971344553 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840d0a50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1971344553 >> v2:172.21.15.112:3300/0 conn(0x7fa8840d1f50 0x7fa8840b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.760+0000 7fa8894b6640 1 -- 172.21.15.105:0/1971344553 >> 172.21.15.105:0/1971344553 conn(0x7fa8840b3e80 msgr2=0x7fa8840b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.761+0000 7f684d8fa640 1 -- 172.21.15.105:0/3581473972 shutdown_connections 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.761+0000 7f684d8fa640 1 --2- 172.21.15.105:0/3581473972 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6848071010 0x7f68480ca760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.761+0000 7f684d8fa640 1 -- 172.21.15.105:0/3581473972 >> 172.21.15.105:0/3581473972 conn(0x7f68480ccb90 msgr2=0x7f68480ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.761+0000 7f684d8fa640 1 -- 172.21.15.105:0/3581473972 shutdown_connections 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.761+0000 7f684d8fa640 1 -- 172.21.15.105:0/3581473972 wait complete. 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/2336015493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ca760 msgr2=0x7f3ae00cb370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 --2- 172.21.15.105:0/2336015493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ca760 0x7f3ae00cb370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c1e0 tx=0x7f3ad402a0c0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7fa8894b6640 1 -- 172.21.15.105:0/1971344553 shutdown_connections 2024-09-21T07:26:40.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/2336015493 shutdown_connections 2024-09-21T07:26:40.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 --2- 172.21.15.105:0/2336015493 >> v2:172.21.15.112:3300/0 conn(0x7f3ae00ce8b0 0x7f3ae00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 --2- 172.21.15.105:0/2336015493 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ca760 0x7f3ae00cb370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/2336015493 >> 172.21.15.105:0/2336015493 conn(0x7f3ae00ccb90 msgr2=0x7f3ae00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.764+0000 7f684d8fa640 1 Processor -- start 2024-09-21T07:26:40.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.763+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/2336015493 shutdown_connections 2024-09-21T07:26:40.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.763+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/2336015493 wait complete. 2024-09-21T07:26:40.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.762+0000 7fa8894b6640 1 -- 172.21.15.105:0/1971344553 wait complete. 2024-09-21T07:26:40.767 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.765+0000 7fa8894b6640 1 Processor -- start 2024-09-21T07:26:40.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.766+0000 7f3ae6d5e640 1 Processor -- start 2024-09-21T07:26:40.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f3ae6d5e640 1 -- start start 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.765+0000 7f684d8fa640 1 -- start start 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 0x7f6848146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 0x7f68480c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68480c4c90 0x7f684809d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6848074170 con 0x7f6848071010 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6848073ff0 con 0x7f68480c4c90 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.767+0000 7f684d8fa640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f68480742f0 con 0x7f68480cb370 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.768+0000 7f6846ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 0x7f6848146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.768+0000 7f68477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68480c4c90 0x7f684809d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.766+0000 7fa8894b6640 1 -- start start 2024-09-21T07:26:40.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.768+0000 7f683e7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 0x7f68480c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.768+0000 7f683e7fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 0x7f68480c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:38988/0 (socket says 172.21.15.105:38988) 2024-09-21T07:26:40.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.768+0000 7f683e7fc640 1 -- 172.21.15.105:0/599941324 learned_addr learned my addr 172.21.15.105:0/599941324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 0x7fa884150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f6846ffd640 1 -- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 msgr2=0x7f68480c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f6846ffd640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 0x7f68480c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f6846ffd640 1 -- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68480c4c90 msgr2=0x7f684809d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f6846ffd640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68480c4c90 0x7f684809d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f6846ffd640 1 -- 172.21.15.105:0/599941324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6848147480 con 0x7f6848071010 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3ae6d5e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ae006ef50 0x7f3ae00c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3ae6d5e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3ae6d5e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 0x7f3ae00c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3ae6d5e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae006fa70 con 0x7f3ae006ef50 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3adffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3adffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:60962/0 (socket says 172.21.15.105:60962) 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7f3adffff640 1 -- 172.21.15.105:0/1422449957 learned_addr learned my addr 172.21.15.105:0/1422449957 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3ae52fb640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 0x7f3ae00c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae0074590 con 0x7f3ae00ca760 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae006fbf0 con 0x7f3ae00ce8b0 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 -- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 msgr2=0x7f3ae00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 0x7f3ae00c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 -- 172.21.15.105:0/1422449957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ae006ef50 msgr2=0x7f3ae00c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ae006ef50 0x7f3ae00c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae00c6a60 con 0x7f3ae00ca760 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3ae52fb640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 0x7f3ae00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa88406f230 con 0x7fa88409ba30 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa88415f4c0 con 0x7fa8840d1f50 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.769+0000 7fa8894b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa8840c9cb0 con 0x7fa88415f0b0 2024-09-21T07:26:40.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7fa8827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7fa8827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:60966/0 (socket says 172.21.15.105:60966) 2024-09-21T07:26:40.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7fa8827fc640 1 -- 172.21.15.105:0/1799057658 learned_addr learned my addr 172.21.15.105:0/1799057658 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.770+0000 7f3adffff640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d770 tx=0x7f3acc00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7fa882ffd640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 0x7fa884150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc017070 con 0x7f3ae00ca760 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae009ab50 con 0x7f3ae00ca760 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3acc009d20 con 0x7f3ae00ca760 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0106c0 con 0x7f3ae00ca760 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7fa8837fe640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7fa8827fc640 1 -- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 msgr2=0x7fa8840cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f6846ffd640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 0x7f6848146f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f683803a000 tx=0x7f6838038b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.772+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683803b070 con 0x7f6848071010 2024-09-21T07:26:40.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7fa8827fc640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7fa8827fc640 1 -- 172.21.15.105:0/1799057658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 msgr2=0x7fa884150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.771+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae00c86b0 con 0x7f3ae00ca760 2024-09-21T07:26:40.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7fa8827fc640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 0x7fa884150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae02069c0 con 0x7f3ae00ca760 2024-09-21T07:26:40.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7fa8827fc640 1 -- 172.21.15.105:0/1799057658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8841512a0 con 0x7fa8840d1f50 2024-09-21T07:26:40.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3acc01a070 con 0x7f3ae00ca760 2024-09-21T07:26:40.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.774+0000 7f3add7fa640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 0x7f3ac407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.774+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3acc09b160 con 0x7f3ae00ca760 2024-09-21T07:26:40.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.775+0000 7fa882ffd640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 0x7fa884150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7f684d8fa640 1 -- 172.21.15.105:0/599941324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68481f5320 con 0x7f6848071010 2024-09-21T07:26:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.773+0000 7f684d8fa640 1 -- 172.21.15.105:0/599941324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68480c8f60 con 0x7f6848071010 2024-09-21T07:26:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.774+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6838005330 con 0x7f6848071010 2024-09-21T07:26:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.774+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838003800 con 0x7f6848071010 2024-09-21T07:26:40.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.775+0000 7f3ae4afa640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 0x7f3ac407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.776+0000 7fa8827fc640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c460 tx=0x7fa87800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.776+0000 7f3ae4afa640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 0x7f3ac407b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4033000 tx=0x7f3ad40046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.776+0000 7f684d8fa640 1 -- 172.21.15.105:0/599941324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68480ca760 con 0x7f6848071010 2024-09-21T07:26:40.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.777+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f683803e070 con 0x7f6848071010 2024-09-21T07:26:40.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.777+0000 7f6844ff9640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 0x7f682807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.777+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f68380c3ac0 con 0x7f6848071010 2024-09-21T07:26:40.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.778+0000 7f683e7fc640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 0x7f682807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.778+0000 7f683e7fc640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 0x7f682807b0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68300046b0 tx=0x7f6830007f50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.779+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780052d0 con 0x7fa8840d1f50 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.779+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa878005470 con 0x7fa8840d1f50 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.779+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa884151580 con 0x7fa8840d1f50 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.780+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa884151b40 con 0x7fa8840d1f50 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.781+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878012890 con 0x7fa8840d1f50 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.779+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ae02069c0 con 0x7f3ae00ca760 2024-09-21T07:26:40.783 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.782+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa864005290 con 0x7fa8840d1f50 2024-09-21T07:26:40.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.784+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa878012a30 con 0x7fa8840d1f50 2024-09-21T07:26:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.785+0000 7fa86b7fe640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 0x7fa87007b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.785+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa87809ae70 con 0x7fa8840d1f50 2024-09-21T07:26:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.785+0000 7fa882ffd640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 0x7fa87007b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.786+0000 7fa882ffd640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 0x7fa87007b110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa874005fd0 tx=0x7fa874009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.787+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6848147480 con 0x7f6848071010 2024-09-21T07:26:40.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.789+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8780630a0 con 0x7fa8840d1f50 2024-09-21T07:26:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.826+0000 7ff24464d640 1 -- 172.21.15.105:0/2354114289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c162b80 msgr2=0x7ff23c06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.826+0000 7ff24464d640 1 --2- 172.21.15.105:0/2354114289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c162b80 0x7ff23c06f960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0ce470 tx=0x7ff23802a540 comp rx=0 tx=0).stop 2024-09-21T07:26:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 -- 172.21.15.105:0/2354114289 shutdown_connections 2024-09-21T07:26:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 --2- 172.21.15.105:0/2354114289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c162b80 0x7ff23c06f960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 --2- 172.21.15.105:0/2354114289 >> v2:172.21.15.105:3300/0 conn(0x7ff23c0aefa0 0x7ff23c0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 -- 172.21.15.105:0/2354114289 >> 172.21.15.105:0/2354114289 conn(0x7ff23c0ccb90 msgr2=0x7ff23c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 -- 172.21.15.105:0/2354114289 shutdown_connections 2024-09-21T07:26:40.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.827+0000 7ff24464d640 1 -- 172.21.15.105:0/2354114289 wait complete. 2024-09-21T07:26:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.829+0000 7fe4d6ffd640 1 -- 172.21.15.105:0/1258816739 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40047a0 con 0x7fe4d00e0030 2024-09-21T07:26:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/1258816739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d00e0030 msgr2=0x7fe4d0031b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/1258816739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d00e0030 0x7fe4d0031b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4005fd0 tx=0x7fe4c402a540 comp rx=0 tx=0).stop 2024-09-21T07:26:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7ff24464d640 1 Processor -- start 2024-09-21T07:26:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7ffb5affd640 1 -- 172.21.15.105:0/2695466306 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035020 con 0x7ffb5c15e920 2024-09-21T07:26:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/1258816739 shutdown_connections 2024-09-21T07:26:40.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/1258816739 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d00533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/1258816739 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d00e0030 0x7fe4d0031b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/1258816739 >> 172.21.15.105:0/1258816739 conn(0x7fe4d001e260 msgr2=0x7fe4d0039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.830+0000 7ff24464d640 1 -- start start 2024-09-21T07:26:40.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/1258816739 shutdown_connections 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 -- 172.21.15.105:0/2695466306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c15e920 msgr2=0x7ffb5c0b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 --2- 172.21.15.105:0/2695466306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c15e920 0x7ffb5c0b3cd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00a940 tx=0x7ffb4c02b910 comp rx=0 tx=0).stop 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 -- 172.21.15.105:0/2695466306 shutdown_connections 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 --2- 172.21.15.105:0/2695466306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c15e920 0x7ffb5c0b3cd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 -- 172.21.15.105:0/2695466306 >> 172.21.15.105:0/2695466306 conn(0x7ffb5c06cda0 msgr2=0x7ffb5c06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 -- 172.21.15.105:0/2695466306 shutdown_connections 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7ffb626dc640 1 -- 172.21.15.105:0/2695466306 wait complete. 2024-09-21T07:26:40.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.831+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/1258816739 wait complete. 2024-09-21T07:26:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.833+0000 7ff24464d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.833+0000 7ff24464d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c0ca970 0x7ff23c09ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.833+0000 7ff2423e9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.833+0000 7ff2423e9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60674/0 (socket says 172.21.15.105:60674) 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.833+0000 7ff2423e9640 1 -- 172.21.15.105:0/1196063786 learned_addr learned my addr 172.21.15.105:0/1196063786 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff241be8640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c0ca970 0x7ff23c09ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 -- 172.21.15.105:0/1690263895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2428162a70 msgr2=0x7f2428162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 --2- 172.21.15.105:0/1690263895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2428162a70 0x7f2428162e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2418009830 tx=0x7f241802a510 comp rx=0 tx=0).stop 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff24464d640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 0x7ff23c1f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 -- 172.21.15.105:0/1690263895 shutdown_connections 2024-09-21T07:26:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff24464d640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0ce900 con 0x7ff23c0aefa0 2024-09-21T07:26:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff24464d640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0ce780 con 0x7ff23c0ca970 2024-09-21T07:26:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff242bea640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 0x7ff23c1f7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ffb626dc640 1 Processor -- start 2024-09-21T07:26:40.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ff24464d640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c0cea80 con 0x7ff23c09cf80 2024-09-21T07:26:40.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 --2- 172.21.15.105:0/1690263895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2428162a70 0x7f2428162e80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 --2- 172.21.15.105:0/1690263895 >> v2:172.21.15.112:3300/0 conn(0x7f242806d4b0 0x7f242815efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 -- 172.21.15.105:0/1690263895 >> 172.21.15.105:0/1690263895 conn(0x7f24280cdce0 msgr2=0x7f24280ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7ffb626dc640 1 -- start start 2024-09-21T07:26:40.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7f243063f640 1 -- 172.21.15.105:0/1690263895 shutdown_connections 2024-09-21T07:26:40.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7f243063f640 1 -- 172.21.15.105:0/1690263895 wait complete. 2024-09-21T07:26:40.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7fe4dd7a8640 1 Processor -- start 2024-09-21T07:26:40.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.834+0000 7fe4dd7a8640 1 -- start start 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 -- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 msgr2=0x7ff23c1f7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 0x7ff23c1f7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 -- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c0ca970 msgr2=0x7ff23c09ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c0ca970 0x7ff23c09ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c09d390 con 0x7ff23c0aefa0 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff242bea640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 0x7ff23c1f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:40.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.836+0000 7fe4dd7a8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4dd7a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 0x7fe4d004c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 0x7ffb5c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb5c15e920 0x7ffb5c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c0d0930 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5c0d07b0 con 0x7ffb5c0b4210 2024-09-21T07:26:40.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb626dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5c0d0ab0 con 0x7ffb5c15e920 2024-09-21T07:26:40.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4d7fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:39036/0 (socket says 172.21.15.105:39036) 2024-09-21T07:26:40.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4d7fff640 1 -- 172.21.15.105:0/4082346515 learned_addr learned my addr 172.21.15.105:0/4082346515 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb5b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 0x7ffb5c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.835+0000 7ff2423e9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c400 tx=0x7ff22c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb5bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.105:32790/0 (socket says 172.21.15.105:32790) 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ffb5bfff640 1 -- 172.21.15.105:0/690950226 learned_addr learned my addr 172.21.15.105:0/690950226 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 -- 172.21.15.105:0/980065988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040ce8b0 msgr2=0x7fce040cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 --2- 172.21.15.105:0/980065988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040ce8b0 0x7fce040cecc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fce04072ed0 tx=0x7fcdfc02b830 comp rx=0 tx=0).stop 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 -- 172.21.15.105:0/980065988 shutdown_connections 2024-09-21T07:26:40.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 --2- 172.21.15.105:0/980065988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce0406ef50 0x7fce0406f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 --2- 172.21.15.105:0/980065988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040ce8b0 0x7fce040cecc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 --2- 172.21.15.105:0/980065988 >> v2:172.21.15.105:3300/0 conn(0x7fce040ca760 0x7fce040cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 -- 172.21.15.105:0/980065988 >> 172.21.15.105:0/980065988 conn(0x7fce040ccb90 msgr2=0x7fce040ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 0x7fe4d00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.836+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011790 con 0x7ff23c0aefa0 2024-09-21T07:26:40.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7f243063f640 1 Processor -- start 2024-09-21T07:26:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.836+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff22c017070 con 0x7ff23c0aefa0 2024-09-21T07:26:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.836+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22c011ce0 con 0x7ff23c0aefa0 2024-09-21T07:26:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0037260 con 0x7fe4d0053910 2024-09-21T07:26:40.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7ff24464d640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c15af30 con 0x7ff23c0aefa0 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d00370e0 con 0x7fe4d00e0030 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4dcfa7640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 0x7fe4d00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 -- 172.21.15.105:0/980065988 shutdown_connections 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fce0a40a640 1 -- 172.21.15.105:0/980065988 wait complete. 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb60c79640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb5c15e920 0x7ffb5c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.846 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 -- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb5c15e920 msgr2=0x7ffb5c156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb5c15e920 0x7ffb5c156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 -- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 msgr2=0x7ffb5c15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 -- 172.21.15.105:0/690950226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5c1f9130 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5bfff640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d00373e0 con 0x7fe4d0052fc0 2024-09-21T07:26:40.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7f243063f640 1 -- start start 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ffb5b7fe640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 0x7ffb5c159580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c033000 tx=0x7ffb4c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.838+0000 7ff24464d640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c09b800 con 0x7ff23c0aefa0 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c0af3b0 con 0x7ff23c0aefa0 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c035020 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb4c003d10 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5c1f9410 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb5c1f99d0 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.837+0000 7fe4d77fe640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 0x7fe4d004c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 -- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 msgr2=0x7fe4d004aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 -- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 msgr2=0x7fe4d00ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 0x7fe4d00ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d00c9cd0 con 0x7fe4d0053910 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.839+0000 7fe4d77fe640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 0x7fe4d004c000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d6b0 tx=0x7fe4cc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4d7fff640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4dcfa7640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 0x7fe4d00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x7fe4d0053910 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4cc005310 con 0x7fe4d0053910 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0097c0 con 0x7fe4d0053910 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d00e1f10 con 0x7fe4d0053910 2024-09-21T07:26:40.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d00c9f70 con 0x7fe4d0053910 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c005280 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7fce0a40a640 1 Processor -- start 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7fce0a40a640 1 -- start start 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce0a40a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce0a40a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b8005290 con 0x7fe4d0053910 2024-09-21T07:26:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce0a40a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 0x7fce04154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c005290 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce037fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7f243063f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24280701e0 0x7f242815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7f243063f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428162a70 0x7f24280c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.840+0000 7f243063f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f243063f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2428073f70 con 0x7f2428162a70 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f243063f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2428163650 con 0x7f24280701e0 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f243063f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f24280740f0 con 0x7f24280c9850 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242ebdc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242ebdc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:39052/0 (socket says 172.21.15.105:39052) 2024-09-21T07:26:40.853 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242ebdc640 1 -- 172.21.15.105:0/950207569 learned_addr learned my addr 172.21.15.105:0/950207569 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242e3db640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24280701e0 0x7f242815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242ebdc640 1 -- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24280701e0 msgr2=0x7f242815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.841+0000 7f242ebdc640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24280701e0 0x7f242815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7f242ebdc640 1 -- 172.21.15.105:0/950207569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428162a70 msgr2=0x7f24280c8310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7f242ebdc640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428162a70 0x7f24280c8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7f242ebdc640 1 -- 172.21.15.105:0/950207569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280b3a90 con 0x7f24280c9850 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7f242ebdc640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f241802ab00 tx=0x7f24180045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.854 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce037fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:39068/0 (socket says 172.21.15.105:39068) 2024-09-21T07:26:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce03fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce03fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60696/0 (socket says 172.21.15.105:60696) 2024-09-21T07:26:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418031b50 con 0x7f24280c9850 2024-09-21T07:26:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.842+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff22c01c460 con 0x7ff23c0aefa0 2024-09-21T07:26:40.855 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7ff232ffd640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 0x7ff22007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff22c09b8d0 con 0x7ff23c0aefa0 2024-09-21T07:26:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7fce037fe640 1 -- 172.21.15.105:0/266122466 learned_addr learned my addr 172.21.15.105:0/266122466 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2418031de0 con 0x7f24280c9850 2024-09-21T07:26:40.856 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406fa20 con 0x7fce0406ef50 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.843+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24280b37b0 con 0x7f24280c9850 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24280b1040 con 0x7f24280c9850 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418005280 con 0x7f24280c9850 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce0406f8a0 con 0x7fce040c6880 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fce0406fba0 con 0x7fce040ca760 2024-09-21T07:26:40.857 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 0x7fce04154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7ff241be8640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 0x7ff22007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 -- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 msgr2=0x7fce041fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 -- 172.21.15.105:0/266122466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 msgr2=0x7fce04153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:40.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7ff241be8640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 0x7ff22007b170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff238034000 tx=0x7ff238004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:40.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.844+0000 7fce089a7640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce040c4a20 con 0x7fce040c6880 2024-09-21T07:26:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce037fe640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce03fff640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce089a7640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 0x7fce04154ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b390 tx=0x7fcdf800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242809ef20 con 0x7f24280c9850 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8015020 con 0x7fce040c6880 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdf8011940 con 0x7fce040c6880 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce040c4d00 con 0x7fce040c6880 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce04154140 con 0x7fce040c6880 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.845+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8011d90 con 0x7fce040c6880 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.846+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2418005420 con 0x7f24280c9850 2024-09-21T07:26:40.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.847+0000 7f241effd640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 0x7f240c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.847+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde4005290 con 0x7fce040c6880 2024-09-21T07:26:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.847+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f24180cd510 con 0x7f24280c9850 2024-09-21T07:26:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.847+0000 7f242e3db640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 0x7f240c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.848+0000 7f242e3db640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 0x7f240c07b170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24280b6280 tx=0x7f2424009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.848+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdf8018020 con 0x7fce040c6880 2024-09-21T07:26:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.848+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff23c09d390 con 0x7ff23c0aefa0 2024-09-21T07:26:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.848+0000 7fce00ff9640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 0x7fcdf007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.848+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcdf809ad90 con 0x7fce040c6880 2024-09-21T07:26:40.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.849+0000 7fce03fff640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 0x7fcdf007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.850+0000 7fce03fff640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 0x7fcdf007b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ea0 tx=0x7fcdf4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.850+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb4c038020 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.850+0000 7ffb58ff9640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 0x7ffb4807b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.851+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ffb4c0bc8a0 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.851+0000 7ffb5bfff640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 0x7ffb4807b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.851+0000 7ffb5bfff640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 0x7ffb4807b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb50007d20 tx=0x7ffb50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.851+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4cc004020 con 0x7fe4d0053910 2024-09-21T07:26:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.852+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2418050030 con 0x7f24280c9850 2024-09-21T07:26:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.852+0000 7fe4d4ff9640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 0x7fe4bc07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:40.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.852+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe4cc09b3a0 con 0x7fe4d0053910 2024-09-21T07:26:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.852+0000 7fe4d7fff640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 0x7fe4bc07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.853+0000 7fe4d7fff640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 0x7fe4bc07b170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40096c0 tx=0x7fe4c4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.853+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcdf801f020 con 0x7fce040c6880 2024-09-21T07:26:40.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.860+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb4c084aa0 con 0x7ffb5c0b49c0 2024-09-21T07:26:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:40.874+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4cc0635a0 con 0x7fe4d0053910 2024-09-21T07:26:41.198 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.196+0000 7f3ae6d5e640 1 -- 172.21.15.105:0/1422449957 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3ae00cb370 con 0x7f3ae00ca760 2024-09-21T07:26:41.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.197+0000 7f3add7fa640 1 -- 172.21.15.105:0/1422449957 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3acc062c10 con 0x7f3ae00ca760 2024-09-21T07:26:41.199 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-09-21T07:26:41.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 msgr2=0x7f3ac407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 0x7f3ac407b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4033000 tx=0x7f3ad40046c0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 msgr2=0x7f3ae00c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d770 tx=0x7f3acc00dc30 comp rx=0 tx=0).stop 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 shutdown_connections 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3ac4078bc0 0x7f3ac407b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ae00ca760 0x7f3ae00c4320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.201+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ae00ce8b0 0x7f3ae00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.202+0000 7f3acaffd640 1 --2- 172.21.15.105:0/1422449957 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ae006ef50 0x7f3ae00c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.202+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 >> 172.21.15.105:0/1422449957 conn(0x7f3ae00ccb90 msgr2=0x7f3ae00ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.202+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 shutdown_connections 2024-09-21T07:26:41.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.202+0000 7f3acaffd640 1 -- 172.21.15.105:0/1422449957 wait complete. 2024-09-21T07:26:41.209 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.207+0000 7f684d8fa640 1 -- 172.21.15.105:0/599941324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f684809e1a0 con 0x7f6848071010 2024-09-21T07:26:41.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.208+0000 7f6844ff9640 1 -- 172.21.15.105:0/599941324 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f683808b6e0 con 0x7f6848071010 2024-09-21T07:26:41.210 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-09-21T07:26:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.212+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 msgr2=0x7f682807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.212+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 0x7f682807b0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f68300046b0 tx=0x7f6830007f50 comp rx=0 tx=0).stop 2024-09-21T07:26:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.212+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 msgr2=0x7f6848146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.212+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 0x7f6848146f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f683803a000 tx=0x7f6838038b00 comp rx=0 tx=0).stop 2024-09-21T07:26:41.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.212+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa864005850 con 0x7fa8840d1f50 2024-09-21T07:26:41.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7fa86b7fe640 1 -- 172.21.15.105:0/1799057658 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8780629a0 con 0x7fa8840d1f50 2024-09-21T07:26:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 shutdown_connections 2024-09-21T07:26:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6828078be0 0x7f682807b0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68480c4c90 0x7f684809d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68480cb370 0x7f68480c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 --2- 172.21.15.105:0/599941324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6848071010 0x7f6848146f40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 >> 172.21.15.105:0/599941324 conn(0x7f68480ccb90 msgr2=0x7f684815a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 shutdown_connections 2024-09-21T07:26:41.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.214+0000 7f683dffb640 1 -- 172.21.15.105:0/599941324 wait complete. 2024-09-21T07:26:41.217 INFO:teuthology.orchestra.run.smithi105.stdout:55834574858 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.217+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 msgr2=0x7fa87007b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.217+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 0x7fa87007b110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa874005fd0 tx=0x7fa874009500 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.217+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 msgr2=0x7fa884159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.217+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c460 tx=0x7fa87800c920 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 shutdown_connections 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa870078c00 0x7fa87007b110 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8840d1f50 0x7fa884159400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa88415f0b0 0x7fa8840cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 --2- 172.21.15.105:0/1799057658 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa88409ba30 0x7fa884150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 >> 172.21.15.105:0/1799057658 conn(0x7fa8840b3e80 msgr2=0x7fa88415a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 shutdown_connections 2024-09-21T07:26:41.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.218+0000 7fa8894b6640 1 -- 172.21.15.105:0/1799057658 wait complete. 2024-09-21T07:26:41.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.5 2024-09-21T07:26:41.222 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.7 2024-09-21T07:26:41.234 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-09-21T07:26:41.234 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.248 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.246+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff23c074e70 con 0x7ff23c0aefa0 2024-09-21T07:26:41.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.247+0000 7ff232ffd640 1 -- 172.21.15.105:0/1196063786 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff22c063ad0 con 0x7ff23c0aefa0 2024-09-21T07:26:41.249 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 msgr2=0x7ff22007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 0x7ff22007b170 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff238034000 tx=0x7ff238004730 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 msgr2=0x7ff23c1f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00c400 tx=0x7ff22c00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 shutdown_connections 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff220078c60 0x7ff22007b170 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff23c0ca970 0x7ff23c09ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff23c09cf80 0x7ff23c1f7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 --2- 172.21.15.105:0/1196063786 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff23c0aefa0 0x7ff23c1f9000 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.252 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.249+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 >> 172.21.15.105:0/1196063786 conn(0x7ff23c0ccb90 msgr2=0x7ff23c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.250+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 shutdown_connections 2024-09-21T07:26:41.253 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.250+0000 7ff230ff9640 1 -- 172.21.15.105:0/1196063786 wait complete. 2024-09-21T07:26:41.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509450 for osd.4 2024-09-21T07:26:41.262 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.268+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f24280b58f0 con 0x7f24280c9850 2024-09-21T07:26:41.271 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.269+0000 7f241effd640 1 -- 172.21.15.105:0/950207569 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f24180955f0 con 0x7f24280c9850 2024-09-21T07:26:41.271 INFO:teuthology.orchestra.run.smithi105.stdout:64424509452 2024-09-21T07:26:41.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 msgr2=0x7f240c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 0x7f240c07b170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f24280b6280 tx=0x7f2424009040 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 msgr2=0x7f24280c8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f241802ab00 tx=0x7f24180045c0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 shutdown_connections 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f240c078c60 0x7f240c07b170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24280c9850 0x7f24280c8850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2428162a70 0x7f24280c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 --2- 172.21.15.105:0/950207569 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24280701e0 0x7f242815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 >> 172.21.15.105:0/950207569 conn(0x7f24280cdce0 msgr2=0x7f24280aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 shutdown_connections 2024-09-21T07:26:41.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.272+0000 7f243063f640 1 -- 172.21.15.105:0/950207569 wait complete. 2024-09-21T07:26:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.283+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fcde4005580 con 0x7fce040c6880 2024-09-21T07:26:41.286 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.283+0000 7fce00ff9640 1 -- 172.21.15.105:0/266122466 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcdf8062820 con 0x7fce040c6880 2024-09-21T07:26:41.287 INFO:teuthology.orchestra.run.smithi105.stdout:64424509451 2024-09-21T07:26:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.286+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 msgr2=0x7fcdf007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.286+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 0x7fcdf007b170 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009ea0 tx=0x7fcdf4005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 msgr2=0x7fce04154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 0x7fce04154ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcdf800b390 tx=0x7fcdf800b850 comp rx=0 tx=0).stop 2024-09-21T07:26:41.289 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 shutdown_connections 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcdf0078c60 0x7fcdf007b170 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce040c6880 0x7fce04154ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fce040ca760 0x7fce041fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 --2- 172.21.15.105:0/266122466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fce0406ef50 0x7fce04153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 >> 172.21.15.105:0/266122466 conn(0x7fce040ccb90 msgr2=0x7fce041645b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 shutdown_connections 2024-09-21T07:26:41.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.287+0000 7fce0a40a640 1 -- 172.21.15.105:0/266122466 wait complete. 2024-09-21T07:26:41.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.1 2024-09-21T07:26:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.289+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffb3c005850 con 0x7ffb5c0b49c0 2024-09-21T07:26:41.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.289+0000 7ffb58ff9640 1 -- 172.21.15.105:0/690950226 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffb4c0843a0 con 0x7ffb5c0b49c0 2024-09-21T07:26:41.291 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.292 INFO:teuthology.orchestra.run.smithi105.stdout:64424509452 2024-09-21T07:26:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 msgr2=0x7ffb4807b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 0x7ffb4807b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffb50007d20 tx=0x7ffb50009040 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 msgr2=0x7ffb5c159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 0x7ffb5c159580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c033000 tx=0x7ffb4c003820 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 shutdown_connections 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffb48078c60 0x7ffb4807b170 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffb5c15e920 0x7ffb5c156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb5c0b49c0 0x7ffb5c159580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.291+0000 7ffb626dc640 1 --2- 172.21.15.105:0/690950226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffb5c0b4210 0x7ffb5c15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.292+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 >> 172.21.15.105:0/690950226 conn(0x7ffb5c06cda0 msgr2=0x7ffb5c15a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.292+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 shutdown_connections 2024-09-21T07:26:41.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.292+0000 7ffb626dc640 1 -- 172.21.15.105:0/690950226 wait complete. 2024-09-21T07:26:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.301+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe4b8005850 con 0x7fe4d0053910 2024-09-21T07:26:41.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.302+0000 7fe4d4ff9640 1 -- 172.21.15.105:0/4082346515 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe4cc062ea0 con 0x7fe4d0053910 2024-09-21T07:26:41.304 INFO:teuthology.orchestra.run.smithi105.stdout:64424509450 2024-09-21T07:26:41.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.0 2024-09-21T07:26:41.306 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.2 2024-09-21T07:26:41.306 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 msgr2=0x7fe4bc07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 0x7fe4bc07b170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40096c0 tx=0x7fe4c4004730 comp rx=0 tx=0).stop 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 msgr2=0x7fe4d004c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 0x7fe4d004c000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d6b0 tx=0x7fe4cc00db70 comp rx=0 tx=0).stop 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 shutdown_connections 2024-09-21T07:26:41.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe4bc078c60 0x7fe4bc07b170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d00e0030 0x7fe4d00ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4d0053910 0x7fe4d004c000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 --2- 172.21.15.105:0/4082346515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe4d0052fc0 0x7fe4d004aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 >> 172.21.15.105:0/4082346515 conn(0x7fe4d001e260 msgr2=0x7fe4d0006da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 shutdown_connections 2024-09-21T07:26:41.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.304+0000 7fe4dd7a8640 1 -- 172.21.15.105:0/4082346515 wait complete. 2024-09-21T07:26:41.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509450 for osd.6 2024-09-21T07:26:41.318 DEBUG:teuthology.parallel:result is None 2024-09-21T07:26:41.318 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:26:41.318 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 -- 172.21.15.105:0/95105851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0cf1b0 msgr2=0x7f508c0cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 --2- 172.21.15.105:0/95105851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0cf1b0 0x7f508c0cf5c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f508c08d170 tx=0x7f508402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 -- 172.21.15.105:0/95105851 shutdown_connections 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 --2- 172.21.15.105:0/95105851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0cf1b0 0x7f508c0cf5c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 --2- 172.21.15.105:0/95105851 >> v2:172.21.15.112:3300/0 conn(0x7f508c0d4460 0x7f508c0d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 -- 172.21.15.105:0/95105851 >> 172.21.15.105:0/95105851 conn(0x7f508c08bfb0 msgr2=0x7f508c09d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 -- 172.21.15.105:0/95105851 shutdown_connections 2024-09-21T07:26:41.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.467+0000 7f50919da640 1 -- 172.21.15.105:0/95105851 wait complete. 2024-09-21T07:26:41.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.469+0000 7f50919da640 1 Processor -- start 2024-09-21T07:26:41.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.469+0000 7f50919da640 1 -- start start 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 0x7f508c09ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c0d3020 0x7f508c157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0cfb00 con 0x7f508c0d4460 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f508c0b4b30 con 0x7f508c0d3020 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f50919da640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f508c0cfc80 con 0x7f508c0cf1b0 2024-09-21T07:26:41.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60710/0 (socket says 172.21.15.105:60710) 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 -- 172.21.15.105:0/783012562 learned_addr learned my addr 172.21.15.105:0/783012562 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508a7fc640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c0d3020 0x7f508c157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508affd640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 0x7f508c09ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 -- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 msgr2=0x7f508c09ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 0x7f508c09ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 -- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c0d3020 msgr2=0x7f508c157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c0d3020 0x7f508c157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508b7fe640 1 -- 172.21.15.105:0/783012562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c1f9e90 con 0x7f508c0d4460 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.470+0000 7f508affd640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 0x7f508c09ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f508b7fe640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f508402aa90 tx=0x7f5084033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:41.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084031b60 con 0x7f508c0d4460 2024-09-21T07:26:41.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5084031e10 con 0x7f508c0d4460 2024-09-21T07:26:41.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084005260 con 0x7f508c0d4460 2024-09-21T07:26:41.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c09f4a0 con 0x7f508c0d4460 2024-09-21T07:26:41.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.471+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c0a1e10 con 0x7f508c0d4460 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.472+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c0a1b80 con 0x7f508c0d4460 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.472+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5084005400 con 0x7f508c0d4460 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.473+0000 7f5073fff640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 0x7f507807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.473+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f50840bc200 con 0x7f508c0d4460 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.473+0000 7f508affd640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 0x7f507807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.474+0000 7f508affd640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 0x7f507807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f507c007fd0 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:41.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.477+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f508403f030 con 0x7f508c0d4460 2024-09-21T07:26:41.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.651+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f508c0d4870 con 0x7f5078078b90 2024-09-21T07:26:41.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.657+0000 7f5073fff640 1 -- 172.21.15.105:0/783012562 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7f508c0d4870 con 0x7f5078078b90 2024-09-21T07:26:41.661 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:41.664 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 msgr2=0x7f507807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 0x7f507807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f507c007fd0 tx=0x7f507c009040 comp rx=0 tx=0).stop 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 msgr2=0x7f508c1f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f508402aa90 tx=0x7f5084033c10 comp rx=0 tx=0).stop 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 shutdown_connections 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5078078b90 0x7f507807b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c0d3020 0x7f508c157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f508c0d4460 0x7f508c1f9040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 --2- 172.21.15.105:0/783012562 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c0cf1b0 0x7f508c09ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.660+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 >> 172.21.15.105:0/783012562 conn(0x7f508c08bfb0 msgr2=0x7f508c0b8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.661+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 shutdown_connections 2024-09-21T07:26:41.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.661+0000 7f50919da640 1 -- 172.21.15.105:0/783012562 wait complete. 2024-09-21T07:26:41.674 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-21T07:26:40.771809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3514368,"data_stored":2101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003251"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755382+0000","last_change":"2024-09-21T07:26:14.746008+0000","last_active":"2024-09-21T07:26:22.755382+0000","last_peered":"2024-09-21T07:26:22.755382+0000","last_clean":"2024-09-21T07:26:22.755382+0000","last_became_active":"2024-09-21T07:26:14.745685+0000","last_became_peered":"2024-09-21T07:26:14.745685+0000","last_unstale":"2024-09-21T07:26:22.755382+0000","last_undegraded":"2024-09-21T07:26:22.755382+0000","last_fullsized":"2024-09-21T07:26:22.755382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:02.797953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217709+0000","last_change":"2024-09-21T07:26:15.767737+0000","last_active":"2024-09-21T07:26:23.217709+0000","last_peered":"2024-09-21T07:26:23.217709+0000","last_clean":"2024-09-21T07:26:23.217709+0000","last_became_active":"2024-09-21T07:26:15.767112+0000","last_became_peered":"2024-09-21T07:26:15.767112+0000","last_unstale":"2024-09-21T07:26:23.217709+0000","last_undegraded":"2024-09-21T07:26:23.217709+0000","last_fullsized":"2024-09-21T07:26:23.217709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.781454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217413+0000","last_change":"2024-09-21T07:26:20.205421+0000","last_active":"2024-09-21T07:26:23.217413+0000","last_peered":"2024-09-21T07:26:23.217413+0000","last_clean":"2024-09-21T07:26:23.217413+0000","last_became_active":"2024-09-21T07:26:20.204943+0000","last_became_peered":"2024-09-21T07:26:20.204943+0000","last_unstale":"2024-09-21T07:26:23.217413+0000","last_undegraded":"2024-09-21T07:26:23.217413+0000","last_fullsized":"2024-09-21T07:26:23.217413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:55.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:14.738042+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:14.737714+0000","last_became_peered":"2024-09-21T07:26:14.737714+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:43.999074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756065+0000","last_change":"2024-09-21T07:26:15.760507+0000","last_active":"2024-09-21T07:26:22.756065+0000","last_peered":"2024-09-21T07:26:22.756065+0000","last_clean":"2024-09-21T07:26:22.756065+0000","last_became_active":"2024-09-21T07:26:15.759859+0000","last_became_peered":"2024-09-21T07:26:15.759859+0000","last_unstale":"2024-09-21T07:26:22.756065+0000","last_undegraded":"2024-09-21T07:26:22.756065+0000","last_fullsized":"2024-09-21T07:26:22.756065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217096+0000","last_change":"2024-09-21T07:26:19.783194+0000","last_active":"2024-09-21T07:26:23.217096+0000","last_peered":"2024-09-21T07:26:23.217096+0000","last_clean":"2024-09-21T07:26:23.217096+0000","last_became_active":"2024-09-21T07:26:19.782625+0000","last_became_peered":"2024-09-21T07:26:19.782625+0000","last_unstale":"2024-09-21T07:26:23.217096+0000","last_undegraded":"2024-09-21T07:26:23.217096+0000","last_fullsized":"2024-09-21T07:26:23.217096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791768+0000","last_change":"2024-09-21T07:26:14.748310+0000","last_active":"2024-09-21T07:26:21.791768+0000","last_peered":"2024-09-21T07:26:21.791768+0000","last_clean":"2024-09-21T07:26:21.791768+0000","last_became_active":"2024-09-21T07:26:14.747943+0000","last_became_peered":"2024-09-21T07:26:14.747943+0000","last_unstale":"2024-09-21T07:26:21.791768+0000","last_undegraded":"2024-09-21T07:26:21.791768+0000","last_fullsized":"2024-09-21T07:26:21.791768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:12.833502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901017+0000","last_change":"2024-09-21T07:26:15.772109+0000","last_active":"2024-09-21T07:26:22.901017+0000","last_peered":"2024-09-21T07:26:22.901017+0000","last_clean":"2024-09-21T07:26:22.901017+0000","last_became_active":"2024-09-21T07:26:15.771762+0000","last_became_peered":"2024-09-21T07:26:15.771762+0000","last_unstale":"2024-09-21T07:26:22.901017+0000","last_undegraded":"2024-09-21T07:26:22.901017+0000","last_fullsized":"2024-09-21T07:26:22.901017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:14.024694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217587+0000","last_change":"2024-09-21T07:26:19.782636+0000","last_active":"2024-09-21T07:26:23.217587+0000","last_peered":"2024-09-21T07:26:23.217587+0000","last_clean":"2024-09-21T07:26:23.217587+0000","last_became_active":"2024-09-21T07:26:19.782148+0000","last_became_peered":"2024-09-21T07:26:19.782148+0000","last_unstale":"2024-09-21T07:26:23.217587+0000","last_undegraded":"2024-09-21T07:26:23.217587+0000","last_fullsized":"2024-09-21T07:26:23.217587+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:34.102740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791003+0000","last_change":"2024-09-21T07:26:14.756864+0000","last_active":"2024-09-21T07:26:21.791003+0000","last_peered":"2024-09-21T07:26:21.791003+0000","last_clean":"2024-09-21T07:26:21.791003+0000","last_became_active":"2024-09-21T07:26:14.756519+0000","last_became_peered":"2024-09-21T07:26:14.756519+0000","last_unstale":"2024-09-21T07:26:21.791003+0000","last_undegraded":"2024-09-21T07:26:21.791003+0000","last_fullsized":"2024-09-21T07:26:21.791003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.306611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901081+0000","last_change":"2024-09-21T07:26:15.769890+0000","last_active":"2024-09-21T07:26:22.901081+0000","last_peered":"2024-09-21T07:26:22.901081+0000","last_clean":"2024-09-21T07:26:22.901081+0000","last_became_active":"2024-09-21T07:26:15.769124+0000","last_became_peered":"2024-09-21T07:26:15.769124+0000","last_unstale":"2024-09-21T07:26:22.901081+0000","last_undegraded":"2024-09-21T07:26:22.901081+0000","last_fullsized":"2024-09-21T07:26:22.901081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:55.122043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217329+0000","last_change":"2024-09-21T07:26:19.788443+0000","last_active":"2024-09-21T07:26:23.217329+0000","last_peered":"2024-09-21T07:26:23.217329+0000","last_clean":"2024-09-21T07:26:23.217329+0000","last_became_active":"2024-09-21T07:26:19.787922+0000","last_became_peered":"2024-09-21T07:26:19.787922+0000","last_unstale":"2024-09-21T07:26:23.217329+0000","last_undegraded":"2024-09-21T07:26:23.217329+0000","last_fullsized":"2024-09-21T07:26:23.217329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790557+0000","last_change":"2024-09-21T07:26:15.774537+0000","last_active":"2024-09-21T07:26:21.790557+0000","last_peered":"2024-09-21T07:26:21.790557+0000","last_clean":"2024-09-21T07:26:21.790557+0000","last_became_active":"2024-09-21T07:26:15.774215+0000","last_became_peered":"2024-09-21T07:26:15.774215+0000","last_unstale":"2024-09-21T07:26:21.790557+0000","last_undegraded":"2024-09-21T07:26:21.790557+0000","last_fullsized":"2024-09-21T07:26:21.790557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791990+0000","last_change":"2024-09-21T07:26:14.752255+0000","last_active":"2024-09-21T07:26:21.791990+0000","last_peered":"2024-09-21T07:26:21.791990+0000","last_clean":"2024-09-21T07:26:21.791990+0000","last_became_active":"2024-09-21T07:26:14.751809+0000","last_became_peered":"2024-09-21T07:26:14.751809+0000","last_unstale":"2024-09-21T07:26:21.791990+0000","last_undegraded":"2024-09-21T07:26:21.791990+0000","last_fullsized":"2024-09-21T07:26:21.791990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:30.055064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218154+0000","last_change":"2024-09-21T07:26:19.784193+0000","last_active":"2024-09-21T07:26:23.218154+0000","last_peered":"2024-09-21T07:26:23.218154+0000","last_clean":"2024-09-21T07:26:23.218154+0000","last_became_active":"2024-09-21T07:26:19.783666+0000","last_became_peered":"2024-09-21T07:26:19.783666+0000","last_unstale":"2024-09-21T07:26:23.218154+0000","last_undegraded":"2024-09-21T07:26:23.218154+0000","last_fullsized":"2024-09-21T07:26:23.218154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218051+0000","last_change":"2024-09-21T07:26:15.761293+0000","last_active":"2024-09-21T07:26:23.218051+0000","last_peered":"2024-09-21T07:26:23.218051+0000","last_clean":"2024-09-21T07:26:23.218051+0000","last_became_active":"2024-09-21T07:26:15.760959+0000","last_became_peered":"2024-09-21T07:26:15.760959+0000","last_unstale":"2024-09-21T07:26:23.218051+0000","last_undegraded":"2024-09-21T07:26:23.218051+0000","last_fullsized":"2024-09-21T07:26:23.218051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:12.766792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218529+0000","last_change":"2024-09-21T07:26:14.737676+0000","last_active":"2024-09-21T07:26:23.218529+0000","last_peered":"2024-09-21T07:26:23.218529+0000","last_clean":"2024-09-21T07:26:23.218529+0000","last_became_active":"2024-09-21T07:26:14.737367+0000","last_became_peered":"2024-09-21T07:26:14.737367+0000","last_unstale":"2024-09-21T07:26:23.218529+0000","last_undegraded":"2024-09-21T07:26:23.218529+0000","last_fullsized":"2024-09-21T07:26:23.218529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217727+0000","last_change":"2024-09-21T07:26:19.782689+0000","last_active":"2024-09-21T07:26:23.217727+0000","last_peered":"2024-09-21T07:26:23.217727+0000","last_clean":"2024-09-21T07:26:23.217727+0000","last_became_active":"2024-09-21T07:26:19.782215+0000","last_became_peered":"2024-09-21T07:26:19.782215+0000","last_unstale":"2024-09-21T07:26:23.217727+0000","last_undegraded":"2024-09-21T07:26:23.217727+0000","last_fullsized":"2024-09-21T07:26:23.217727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218798+0000","last_change":"2024-09-21T07:26:15.758262+0000","last_active":"2024-09-21T07:26:23.218798+0000","last_peered":"2024-09-21T07:26:23.218798+0000","last_clean":"2024-09-21T07:26:23.218798+0000","last_became_active":"2024-09-21T07:26:15.757944+0000","last_became_peered":"2024-09-21T07:26:15.757944+0000","last_unstale":"2024-09-21T07:26:23.218798+0000","last_undegraded":"2024-09-21T07:26:23.218798+0000","last_fullsized":"2024-09-21T07:26:23.218798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:10.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216341+0000","last_change":"2024-09-21T07:26:14.745239+0000","last_active":"2024-09-21T07:26:23.216341+0000","last_peered":"2024-09-21T07:26:23.216341+0000","last_clean":"2024-09-21T07:26:23.216341+0000","last_became_active":"2024-09-21T07:26:14.744901+0000","last_became_peered":"2024-09-21T07:26:14.744901+0000","last_unstale":"2024-09-21T07:26:23.216341+0000","last_undegraded":"2024-09-21T07:26:23.216341+0000","last_fullsized":"2024-09-21T07:26:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:07.561158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218435+0000","last_change":"2024-09-21T07:26:20.205294+0000","last_active":"2024-09-21T07:26:23.218435+0000","last_peered":"2024-09-21T07:26:23.218435+0000","last_clean":"2024-09-21T07:26:23.218435+0000","last_became_active":"2024-09-21T07:26:20.204855+0000","last_became_peered":"2024-09-21T07:26:20.204855+0000","last_unstale":"2024-09-21T07:26:23.218435+0000","last_undegraded":"2024-09-21T07:26:23.218435+0000","last_fullsized":"2024-09-21T07:26:23.218435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217428+0000","last_change":"2024-09-21T07:26:15.762544+0000","last_active":"2024-09-21T07:26:23.217428+0000","last_peered":"2024-09-21T07:26:23.217428+0000","last_clean":"2024-09-21T07:26:23.217428+0000","last_became_active":"2024-09-21T07:26:15.762193+0000","last_became_peered":"2024-09-21T07:26:15.762193+0000","last_unstale":"2024-09-21T07:26:23.217428+0000","last_undegraded":"2024-09-21T07:26:23.217428+0000","last_fullsized":"2024-09-21T07:26:23.217428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:42.681607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790143+0000","last_change":"2024-09-21T07:26:14.748084+0000","last_active":"2024-09-21T07:26:21.790143+0000","last_peered":"2024-09-21T07:26:21.790143+0000","last_clean":"2024-09-21T07:26:21.790143+0000","last_became_active":"2024-09-21T07:26:14.747288+0000","last_became_peered":"2024-09-21T07:26:14.747288+0000","last_unstale":"2024-09-21T07:26:21.790143+0000","last_undegraded":"2024-09-21T07:26:21.790143+0000","last_fullsized":"2024-09-21T07:26:21.790143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:39.950583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:19.788640+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:19.788072+0000","last_became_peered":"2024-09-21T07:26:19.788072+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756286+0000","last_change":"2024-09-21T07:26:15.757686+0000","last_active":"2024-09-21T07:26:22.756286+0000","last_peered":"2024-09-21T07:26:22.756286+0000","last_clean":"2024-09-21T07:26:22.756286+0000","last_became_active":"2024-09-21T07:26:15.757361+0000","last_became_peered":"2024-09-21T07:26:15.757361+0000","last_unstale":"2024-09-21T07:26:22.756286+0000","last_undegraded":"2024-09-21T07:26:22.756286+0000","last_fullsized":"2024-09-21T07:26:22.756286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:49.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791712+0000","last_change":"2024-09-21T07:26:14.746130+0000","last_active":"2024-09-21T07:26:21.791712+0000","last_peered":"2024-09-21T07:26:21.791712+0000","last_clean":"2024-09-21T07:26:21.791712+0000","last_became_active":"2024-09-21T07:26:14.745763+0000","last_became_peered":"2024-09-21T07:26:14.745763+0000","last_unstale":"2024-09-21T07:26:21.791712+0000","last_undegraded":"2024-09-21T07:26:21.791712+0000","last_fullsized":"2024-09-21T07:26:21.791712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217052+0000","last_change":"2024-09-21T07:26:20.203703+0000","last_active":"2024-09-21T07:26:23.217052+0000","last_peered":"2024-09-21T07:26:23.217052+0000","last_clean":"2024-09-21T07:26:23.217052+0000","last_became_active":"2024-09-21T07:26:20.202561+0000","last_became_peered":"2024-09-21T07:26:20.202561+0000","last_unstale":"2024-09-21T07:26:23.217052+0000","last_undegraded":"2024-09-21T07:26:23.217052+0000","last_fullsized":"2024-09-21T07:26:23.217052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:27.041439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218313+0000","last_change":"2024-09-21T07:26:15.768378+0000","last_active":"2024-09-21T07:26:23.218313+0000","last_peered":"2024-09-21T07:26:23.218313+0000","last_clean":"2024-09-21T07:26:23.218313+0000","last_became_active":"2024-09-21T07:26:15.768068+0000","last_became_peered":"2024-09-21T07:26:15.768068+0000","last_unstale":"2024-09-21T07:26:23.218313+0000","last_undegraded":"2024-09-21T07:26:23.218313+0000","last_fullsized":"2024-09-21T07:26:23.218313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:12.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216461+0000","last_change":"2024-09-21T07:26:14.741727+0000","last_active":"2024-09-21T07:26:23.216461+0000","last_peered":"2024-09-21T07:26:23.216461+0000","last_clean":"2024-09-21T07:26:23.216461+0000","last_became_active":"2024-09-21T07:26:14.741372+0000","last_became_peered":"2024-09-21T07:26:14.741372+0000","last_unstale":"2024-09-21T07:26:23.216461+0000","last_undegraded":"2024-09-21T07:26:23.216461+0000","last_fullsized":"2024-09-21T07:26:23.216461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:48.123776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218506+0000","last_change":"2024-09-21T07:26:20.203003+0000","last_active":"2024-09-21T07:26:23.218506+0000","last_peered":"2024-09-21T07:26:23.218506+0000","last_clean":"2024-09-21T07:26:23.218506+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:23.218506+0000","last_undegraded":"2024-09-21T07:26:23.218506+0000","last_fullsized":"2024-09-21T07:26:23.218506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:12.749885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217922+0000","last_change":"2024-09-21T07:26:15.750763+0000","last_active":"2024-09-21T07:26:23.217922+0000","last_peered":"2024-09-21T07:26:23.217922+0000","last_clean":"2024-09-21T07:26:23.217922+0000","last_became_active":"2024-09-21T07:26:15.750444+0000","last_became_peered":"2024-09-21T07:26:15.750444+0000","last_unstale":"2024-09-21T07:26:23.217922+0000","last_undegraded":"2024-09-21T07:26:23.217922+0000","last_fullsized":"2024-09-21T07:26:23.217922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792041+0000","last_change":"2024-09-21T07:26:14.745457+0000","last_active":"2024-09-21T07:26:21.792041+0000","last_peered":"2024-09-21T07:26:21.792041+0000","last_clean":"2024-09-21T07:26:21.792041+0000","last_became_active":"2024-09-21T07:26:14.745075+0000","last_became_peered":"2024-09-21T07:26:14.745075+0000","last_unstale":"2024-09-21T07:26:21.792041+0000","last_undegraded":"2024-09-21T07:26:21.792041+0000","last_fullsized":"2024-09-21T07:26:21.792041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:49.174372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756061+0000","last_change":"2024-09-21T07:26:20.202406+0000","last_active":"2024-09-21T07:26:22.756061+0000","last_peered":"2024-09-21T07:26:22.756061+0000","last_clean":"2024-09-21T07:26:22.756061+0000","last_became_active":"2024-09-21T07:26:20.201970+0000","last_became_peered":"2024-09-21T07:26:20.201970+0000","last_unstale":"2024-09-21T07:26:22.756061+0000","last_undegraded":"2024-09-21T07:26:22.756061+0000","last_fullsized":"2024-09-21T07:26:22.756061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901194+0000","last_change":"2024-09-21T07:26:15.769492+0000","last_active":"2024-09-21T07:26:22.901194+0000","last_peered":"2024-09-21T07:26:22.901194+0000","last_clean":"2024-09-21T07:26:22.901194+0000","last_became_active":"2024-09-21T07:26:15.768661+0000","last_became_peered":"2024-09-21T07:26:15.768661+0000","last_unstale":"2024-09-21T07:26:22.901194+0000","last_undegraded":"2024-09-21T07:26:22.901194+0000","last_fullsized":"2024-09-21T07:26:22.901194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:20.108036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218024+0000","last_change":"2024-09-21T07:26:14.737203+0000","last_active":"2024-09-21T07:26:23.218024+0000","last_peered":"2024-09-21T07:26:23.218024+0000","last_clean":"2024-09-21T07:26:23.218024+0000","last_became_active":"2024-09-21T07:26:14.736387+0000","last_became_peered":"2024-09-21T07:26:14.736387+0000","last_unstale":"2024-09-21T07:26:23.218024+0000","last_undegraded":"2024-09-21T07:26:23.218024+0000","last_fullsized":"2024-09-21T07:26:23.218024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217332+0000","last_change":"2024-09-21T07:26:19.793567+0000","last_active":"2024-09-21T07:26:23.217332+0000","last_peered":"2024-09-21T07:26:23.217332+0000","last_clean":"2024-09-21T07:26:23.217332+0000","last_became_active":"2024-09-21T07:26:19.793063+0000","last_became_peered":"2024-09-21T07:26:19.793063+0000","last_unstale":"2024-09-21T07:26:23.217332+0000","last_undegraded":"2024-09-21T07:26:23.217332+0000","last_fullsized":"2024-09-21T07:26:23.217332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:15.764293+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:15.763942+0000","last_became_peered":"2024-09-21T07:26:15.763942+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:40.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900887+0000","last_change":"2024-09-21T07:26:14.753757+0000","last_active":"2024-09-21T07:26:22.900887+0000","last_peered":"2024-09-21T07:26:22.900887+0000","last_clean":"2024-09-21T07:26:22.900887+0000","last_became_active":"2024-09-21T07:26:14.753415+0000","last_became_peered":"2024-09-21T07:26:14.753415+0000","last_unstale":"2024-09-21T07:26:22.900887+0000","last_undegraded":"2024-09-21T07:26:22.900887+0000","last_fullsized":"2024-09-21T07:26:22.900887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:48.397593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217762+0000","last_change":"2024-09-21T07:26:19.786373+0000","last_active":"2024-09-21T07:26:23.217762+0000","last_peered":"2024-09-21T07:26:23.217762+0000","last_clean":"2024-09-21T07:26:23.217762+0000","last_became_active":"2024-09-21T07:26:19.785794+0000","last_became_peered":"2024-09-21T07:26:19.785794+0000","last_unstale":"2024-09-21T07:26:23.217762+0000","last_undegraded":"2024-09-21T07:26:23.217762+0000","last_fullsized":"2024-09-21T07:26:23.217762+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216872+0000","last_change":"2024-09-21T07:26:15.752034+0000","last_active":"2024-09-21T07:26:23.216872+0000","last_peered":"2024-09-21T07:26:23.216872+0000","last_clean":"2024-09-21T07:26:23.216872+0000","last_became_active":"2024-09-21T07:26:15.751733+0000","last_became_peered":"2024-09-21T07:26:15.751733+0000","last_unstale":"2024-09-21T07:26:23.216872+0000","last_undegraded":"2024-09-21T07:26:23.216872+0000","last_fullsized":"2024-09-21T07:26:23.216872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789086+0000","last_change":"2024-09-21T07:26:14.741318+0000","last_active":"2024-09-21T07:26:21.789086+0000","last_peered":"2024-09-21T07:26:21.789086+0000","last_clean":"2024-09-21T07:26:21.789086+0000","last_became_active":"2024-09-21T07:26:14.740828+0000","last_became_peered":"2024-09-21T07:26:14.740828+0000","last_unstale":"2024-09-21T07:26:21.789086+0000","last_undegraded":"2024-09-21T07:26:21.789086+0000","last_fullsized":"2024-09-21T07:26:21.789086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.614543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218456+0000","last_change":"2024-09-21T07:26:19.793288+0000","last_active":"2024-09-21T07:26:23.218456+0000","last_peered":"2024-09-21T07:26:23.218456+0000","last_clean":"2024-09-21T07:26:23.218456+0000","last_became_active":"2024-09-21T07:26:19.792781+0000","last_became_peered":"2024-09-21T07:26:19.792781+0000","last_unstale":"2024-09-21T07:26:23.218456+0000","last_undegraded":"2024-09-21T07:26:23.218456+0000","last_fullsized":"2024-09-21T07:26:23.218456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755586+0000","last_change":"2024-09-21T07:26:15.763289+0000","last_active":"2024-09-21T07:26:22.755586+0000","last_peered":"2024-09-21T07:26:22.755586+0000","last_clean":"2024-09-21T07:26:22.755586+0000","last_became_active":"2024-09-21T07:26:15.762914+0000","last_became_peered":"2024-09-21T07:26:15.762914+0000","last_unstale":"2024-09-21T07:26:22.755586+0000","last_undegraded":"2024-09-21T07:26:22.755586+0000","last_fullsized":"2024-09-21T07:26:22.755586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:01.890170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790269+0000","last_change":"2024-09-21T07:26:14.746676+0000","last_active":"2024-09-21T07:26:21.790269+0000","last_peered":"2024-09-21T07:26:21.790269+0000","last_clean":"2024-09-21T07:26:21.790269+0000","last_became_active":"2024-09-21T07:26:14.745094+0000","last_became_peered":"2024-09-21T07:26:14.745094+0000","last_unstale":"2024-09-21T07:26:21.790269+0000","last_undegraded":"2024-09-21T07:26:21.790269+0000","last_fullsized":"2024-09-21T07:26:21.790269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:48.031389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756099+0000","last_change":"2024-09-21T07:26:20.202272+0000","last_active":"2024-09-21T07:26:22.756099+0000","last_peered":"2024-09-21T07:26:22.756099+0000","last_clean":"2024-09-21T07:26:22.756099+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:22.756099+0000","last_undegraded":"2024-09-21T07:26:22.756099+0000","last_fullsized":"2024-09-21T07:26:22.756099+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217063+0000","last_change":"2024-09-21T07:26:15.778494+0000","last_active":"2024-09-21T07:26:23.217063+0000","last_peered":"2024-09-21T07:26:23.217063+0000","last_clean":"2024-09-21T07:26:23.217063+0000","last_became_active":"2024-09-21T07:26:15.778117+0000","last_became_peered":"2024-09-21T07:26:15.778117+0000","last_unstale":"2024-09-21T07:26:23.217063+0000","last_undegraded":"2024-09-21T07:26:23.217063+0000","last_fullsized":"2024-09-21T07:26:23.217063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:52.614175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900190+0000","last_change":"2024-09-21T07:26:14.755827+0000","last_active":"2024-09-21T07:26:22.900190+0000","last_peered":"2024-09-21T07:26:22.900190+0000","last_clean":"2024-09-21T07:26:22.900190+0000","last_became_active":"2024-09-21T07:26:14.755319+0000","last_became_peered":"2024-09-21T07:26:14.755319+0000","last_unstale":"2024-09-21T07:26:22.900190+0000","last_undegraded":"2024-09-21T07:26:22.900190+0000","last_fullsized":"2024-09-21T07:26:22.900190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:31.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790381+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:26:21.790381+0000","last_peered":"2024-09-21T07:26:21.790381+0000","last_clean":"2024-09-21T07:26:21.790381+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:26:21.790381+0000","last_undegraded":"2024-09-21T07:26:21.790381+0000","last_fullsized":"2024-09-21T07:26:21.790381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790882+0000","last_change":"2024-09-21T07:26:20.205543+0000","last_active":"2024-09-21T07:26:21.790882+0000","last_peered":"2024-09-21T07:26:21.790882+0000","last_clean":"2024-09-21T07:26:21.790882+0000","last_became_active":"2024-09-21T07:26:20.204540+0000","last_became_peered":"2024-09-21T07:26:20.204540+0000","last_unstale":"2024-09-21T07:26:21.790882+0000","last_undegraded":"2024-09-21T07:26:21.790882+0000","last_fullsized":"2024-09-21T07:26:21.790882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791526+0000","last_change":"2024-09-21T07:26:15.776215+0000","last_active":"2024-09-21T07:26:21.791526+0000","last_peered":"2024-09-21T07:26:21.791526+0000","last_clean":"2024-09-21T07:26:21.791526+0000","last_became_active":"2024-09-21T07:26:15.775884+0000","last_became_peered":"2024-09-21T07:26:15.775884+0000","last_unstale":"2024-09-21T07:26:21.791526+0000","last_undegraded":"2024-09-21T07:26:21.791526+0000","last_fullsized":"2024-09-21T07:26:21.791526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:01.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789487+0000","last_change":"2024-09-21T07:26:14.756009+0000","last_active":"2024-09-21T07:26:21.789487+0000","last_peered":"2024-09-21T07:26:21.789487+0000","last_clean":"2024-09-21T07:26:21.789487+0000","last_became_active":"2024-09-21T07:26:14.755653+0000","last_became_peered":"2024-09-21T07:26:14.755653+0000","last_unstale":"2024-09-21T07:26:21.789487+0000","last_undegraded":"2024-09-21T07:26:21.789487+0000","last_fullsized":"2024-09-21T07:26:21.789487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:22.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790353+0000","last_change":"2024-09-21T07:26:20.202159+0000","last_active":"2024-09-21T07:26:21.790353+0000","last_peered":"2024-09-21T07:26:21.790353+0000","last_clean":"2024-09-21T07:26:21.790353+0000","last_became_active":"2024-09-21T07:26:20.201680+0000","last_became_peered":"2024-09-21T07:26:20.201680+0000","last_unstale":"2024-09-21T07:26:21.790353+0000","last_undegraded":"2024-09-21T07:26:21.790353+0000","last_fullsized":"2024-09-21T07:26:21.790353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:56.792767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901045+0000","last_change":"2024-09-21T07:26:15.771710+0000","last_active":"2024-09-21T07:26:22.901045+0000","last_peered":"2024-09-21T07:26:22.901045+0000","last_clean":"2024-09-21T07:26:22.901045+0000","last_became_active":"2024-09-21T07:26:15.770649+0000","last_became_peered":"2024-09-21T07:26:15.770649+0000","last_unstale":"2024-09-21T07:26:22.901045+0000","last_undegraded":"2024-09-21T07:26:22.901045+0000","last_fullsized":"2024-09-21T07:26:22.901045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.904168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218377+0000","last_change":"2024-09-21T07:26:14.740185+0000","last_active":"2024-09-21T07:26:23.218377+0000","last_peered":"2024-09-21T07:26:23.218377+0000","last_clean":"2024-09-21T07:26:23.218377+0000","last_became_active":"2024-09-21T07:26:14.738935+0000","last_became_peered":"2024-09-21T07:26:14.738935+0000","last_unstale":"2024-09-21T07:26:23.218377+0000","last_undegraded":"2024-09-21T07:26:23.218377+0000","last_fullsized":"2024-09-21T07:26:23.218377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756214+0000","last_change":"2024-09-21T07:26:19.789320+0000","last_active":"2024-09-21T07:26:22.756214+0000","last_peered":"2024-09-21T07:26:22.756214+0000","last_clean":"2024-09-21T07:26:22.756214+0000","last_became_active":"2024-09-21T07:26:19.788692+0000","last_became_peered":"2024-09-21T07:26:19.788692+0000","last_unstale":"2024-09-21T07:26:22.756214+0000","last_undegraded":"2024-09-21T07:26:22.756214+0000","last_fullsized":"2024-09-21T07:26:22.756214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792007+0000","last_change":"2024-09-21T07:26:15.775282+0000","last_active":"2024-09-21T07:26:21.792007+0000","last_peered":"2024-09-21T07:26:21.792007+0000","last_clean":"2024-09-21T07:26:21.792007+0000","last_became_active":"2024-09-21T07:26:15.775046+0000","last_became_peered":"2024-09-21T07:26:15.775046+0000","last_unstale":"2024-09-21T07:26:21.792007+0000","last_undegraded":"2024-09-21T07:26:21.792007+0000","last_fullsized":"2024-09-21T07:26:21.792007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:51.588092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789401+0000","last_change":"2024-09-21T07:26:14.740365+0000","last_active":"2024-09-21T07:26:21.789401+0000","last_peered":"2024-09-21T07:26:21.789401+0000","last_clean":"2024-09-21T07:26:21.789401+0000","last_became_active":"2024-09-21T07:26:14.739987+0000","last_became_peered":"2024-09-21T07:26:14.739987+0000","last_unstale":"2024-09-21T07:26:21.789401+0000","last_undegraded":"2024-09-21T07:26:21.789401+0000","last_fullsized":"2024-09-21T07:26:21.789401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900421+0000","last_change":"2024-09-21T07:26:20.205672+0000","last_active":"2024-09-21T07:26:22.900421+0000","last_peered":"2024-09-21T07:26:22.900421+0000","last_clean":"2024-09-21T07:26:22.900421+0000","last_became_active":"2024-09-21T07:26:20.205251+0000","last_became_peered":"2024-09-21T07:26:20.205251+0000","last_unstale":"2024-09-21T07:26:22.900421+0000","last_undegraded":"2024-09-21T07:26:22.900421+0000","last_fullsized":"2024-09-21T07:26:22.900421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:18.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790765+0000","last_change":"2024-09-21T07:26:15.762918+0000","last_active":"2024-09-21T07:26:21.790765+0000","last_peered":"2024-09-21T07:26:21.790765+0000","last_clean":"2024-09-21T07:26:21.790765+0000","last_became_active":"2024-09-21T07:26:15.762580+0000","last_became_peered":"2024-09-21T07:26:15.762580+0000","last_unstale":"2024-09-21T07:26:21.790765+0000","last_undegraded":"2024-09-21T07:26:21.790765+0000","last_fullsized":"2024-09-21T07:26:21.790765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.691687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218021+0000","last_change":"2024-09-21T07:26:14.739242+0000","last_active":"2024-09-21T07:26:23.218021+0000","last_peered":"2024-09-21T07:26:23.218021+0000","last_clean":"2024-09-21T07:26:23.218021+0000","last_became_active":"2024-09-21T07:26:14.737910+0000","last_became_peered":"2024-09-21T07:26:14.737910+0000","last_unstale":"2024-09-21T07:26:23.218021+0000","last_undegraded":"2024-09-21T07:26:23.218021+0000","last_fullsized":"2024-09-21T07:26:23.218021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:30.239449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217037+0000","last_change":"2024-09-21T07:26:19.782735+0000","last_active":"2024-09-21T07:26:23.217037+0000","last_peered":"2024-09-21T07:26:23.217037+0000","last_clean":"2024-09-21T07:26:23.217037+0000","last_became_active":"2024-09-21T07:26:19.782062+0000","last_became_peered":"2024-09-21T07:26:19.782062+0000","last_unstale":"2024-09-21T07:26:23.217037+0000","last_undegraded":"2024-09-21T07:26:23.217037+0000","last_fullsized":"2024-09-21T07:26:23.217037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790993+0000","last_change":"2024-09-21T07:26:15.772188+0000","last_active":"2024-09-21T07:26:21.790993+0000","last_peered":"2024-09-21T07:26:21.790993+0000","last_clean":"2024-09-21T07:26:21.790993+0000","last_became_active":"2024-09-21T07:26:15.771854+0000","last_became_peered":"2024-09-21T07:26:15.771854+0000","last_unstale":"2024-09-21T07:26:21.790993+0000","last_undegraded":"2024-09-21T07:26:21.790993+0000","last_fullsized":"2024-09-21T07:26:21.790993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:52.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.788951+0000","last_change":"2024-09-21T07:26:14.739783+0000","last_active":"2024-09-21T07:26:21.788951+0000","last_peered":"2024-09-21T07:26:21.788951+0000","last_clean":"2024-09-21T07:26:21.788951+0000","last_became_active":"2024-09-21T07:26:14.739433+0000","last_became_peered":"2024-09-21T07:26:14.739433+0000","last_unstale":"2024-09-21T07:26:21.788951+0000","last_undegraded":"2024-09-21T07:26:21.788951+0000","last_fullsized":"2024-09-21T07:26:21.788951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:08.827173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756045+0000","last_change":"2024-09-21T07:26:19.789353+0000","last_active":"2024-09-21T07:26:22.756045+0000","last_peered":"2024-09-21T07:26:22.756045+0000","last_clean":"2024-09-21T07:26:22.756045+0000","last_became_active":"2024-09-21T07:26:19.788890+0000","last_became_peered":"2024-09-21T07:26:19.788890+0000","last_unstale":"2024-09-21T07:26:22.756045+0000","last_undegraded":"2024-09-21T07:26:22.756045+0000","last_fullsized":"2024-09-21T07:26:22.756045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790254+0000","last_change":"2024-09-21T07:26:15.761846+0000","last_active":"2024-09-21T07:26:21.790254+0000","last_peered":"2024-09-21T07:26:21.790254+0000","last_clean":"2024-09-21T07:26:21.790254+0000","last_became_active":"2024-09-21T07:26:15.761125+0000","last_became_peered":"2024-09-21T07:26:15.761125+0000","last_unstale":"2024-09-21T07:26:21.790254+0000","last_undegraded":"2024-09-21T07:26:21.790254+0000","last_fullsized":"2024-09-21T07:26:21.790254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791303+0000","last_change":"2024-09-21T07:26:14.751611+0000","last_active":"2024-09-21T07:26:21.791303+0000","last_peered":"2024-09-21T07:26:21.791303+0000","last_clean":"2024-09-21T07:26:21.791303+0000","last_became_active":"2024-09-21T07:26:14.750246+0000","last_became_peered":"2024-09-21T07:26:14.750246+0000","last_unstale":"2024-09-21T07:26:21.791303+0000","last_undegraded":"2024-09-21T07:26:21.791303+0000","last_fullsized":"2024-09-21T07:26:21.791303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:45.761871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218074+0000","last_change":"2024-09-21T07:26:20.202567+0000","last_active":"2024-09-21T07:26:23.218074+0000","last_peered":"2024-09-21T07:26:23.218074+0000","last_clean":"2024-09-21T07:26:23.218074+0000","last_became_active":"2024-09-21T07:26:20.201637+0000","last_became_peered":"2024-09-21T07:26:20.201637+0000","last_unstale":"2024-09-21T07:26:23.218074+0000","last_undegraded":"2024-09-21T07:26:23.218074+0000","last_fullsized":"2024-09-21T07:26:23.218074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:17.143053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791707+0000","last_change":"2024-09-21T07:26:15.775390+0000","last_active":"2024-09-21T07:26:21.791707+0000","last_peered":"2024-09-21T07:26:21.791707+0000","last_clean":"2024-09-21T07:26:21.791707+0000","last_became_active":"2024-09-21T07:26:15.774972+0000","last_became_peered":"2024-09-21T07:26:15.774972+0000","last_unstale":"2024-09-21T07:26:21.791707+0000","last_undegraded":"2024-09-21T07:26:21.791707+0000","last_fullsized":"2024-09-21T07:26:21.791707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.551786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900819+0000","last_change":"2024-09-21T07:26:14.752809+0000","last_active":"2024-09-21T07:26:22.900819+0000","last_peered":"2024-09-21T07:26:22.900819+0000","last_clean":"2024-09-21T07:26:22.900819+0000","last_became_active":"2024-09-21T07:26:14.752442+0000","last_became_peered":"2024-09-21T07:26:14.752442+0000","last_unstale":"2024-09-21T07:26:22.900819+0000","last_undegraded":"2024-09-21T07:26:22.900819+0000","last_fullsized":"2024-09-21T07:26:22.900819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.370404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218055+0000","last_change":"2024-09-21T07:26:20.202097+0000","last_active":"2024-09-21T07:26:23.218055+0000","last_peered":"2024-09-21T07:26:23.218055+0000","last_clean":"2024-09-21T07:26:23.218055+0000","last_became_active":"2024-09-21T07:26:20.201641+0000","last_became_peered":"2024-09-21T07:26:20.201641+0000","last_unstale":"2024-09-21T07:26:23.218055+0000","last_undegraded":"2024-09-21T07:26:23.218055+0000","last_fullsized":"2024-09-21T07:26:23.218055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:52.838818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755771+0000","last_change":"2024-09-21T07:26:15.763851+0000","last_active":"2024-09-21T07:26:22.755771+0000","last_peered":"2024-09-21T07:26:22.755771+0000","last_clean":"2024-09-21T07:26:22.755771+0000","last_became_active":"2024-09-21T07:26:15.763088+0000","last_became_peered":"2024-09-21T07:26:15.763088+0000","last_unstale":"2024-09-21T07:26:22.755771+0000","last_undegraded":"2024-09-21T07:26:22.755771+0000","last_fullsized":"2024-09-21T07:26:22.755771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:00.411908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756372+0000","last_change":"2024-09-21T07:26:14.749147+0000","last_active":"2024-09-21T07:26:22.756372+0000","last_peered":"2024-09-21T07:26:22.756372+0000","last_clean":"2024-09-21T07:26:22.756372+0000","last_became_active":"2024-09-21T07:26:14.748858+0000","last_became_peered":"2024-09-21T07:26:14.748858+0000","last_unstale":"2024-09-21T07:26:22.756372+0000","last_undegraded":"2024-09-21T07:26:22.756372+0000","last_fullsized":"2024-09-21T07:26:22.756372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:19.994800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901148+0000","last_change":"2024-09-21T07:26:20.202180+0000","last_active":"2024-09-21T07:26:22.901148+0000","last_peered":"2024-09-21T07:26:22.901148+0000","last_clean":"2024-09-21T07:26:22.901148+0000","last_became_active":"2024-09-21T07:26:20.201665+0000","last_became_peered":"2024-09-21T07:26:20.201665+0000","last_unstale":"2024-09-21T07:26:22.901148+0000","last_undegraded":"2024-09-21T07:26:22.901148+0000","last_fullsized":"2024-09-21T07:26:22.901148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:49.726125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.763407+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.763087+0000","last_became_peered":"2024-09-21T07:26:15.763087+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:49.061741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:26.613007+0000","last_change":"2024-09-21T07:26:14.746399+0000","last_active":"2024-09-21T07:26:26.613007+0000","last_peered":"2024-09-21T07:26:26.613007+0000","last_clean":"2024-09-21T07:26:26.613007+0000","last_became_active":"2024-09-21T07:26:14.746029+0000","last_became_peered":"2024-09-21T07:26:14.746029+0000","last_unstale":"2024-09-21T07:26:26.613007+0000","last_undegraded":"2024-09-21T07:26:26.613007+0000","last_fullsized":"2024-09-21T07:26:26.613007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:47.548805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217015+0000","last_change":"2024-09-21T07:26:19.793971+0000","last_active":"2024-09-21T07:26:23.217015+0000","last_peered":"2024-09-21T07:26:23.217015+0000","last_clean":"2024-09-21T07:26:23.217015+0000","last_became_active":"2024-09-21T07:26:19.793342+0000","last_became_peered":"2024-09-21T07:26:19.793342+0000","last_unstale":"2024-09-21T07:26:23.217015+0000","last_undegraded":"2024-09-21T07:26:23.217015+0000","last_fullsized":"2024-09-21T07:26:23.217015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217997+0000","last_change":"2024-09-21T07:26:15.763698+0000","last_active":"2024-09-21T07:26:23.217997+0000","last_peered":"2024-09-21T07:26:23.217997+0000","last_clean":"2024-09-21T07:26:23.217997+0000","last_became_active":"2024-09-21T07:26:15.763010+0000","last_became_peered":"2024-09-21T07:26:15.763010+0000","last_unstale":"2024-09-21T07:26:23.217997+0000","last_undegraded":"2024-09-21T07:26:23.217997+0000","last_fullsized":"2024-09-21T07:26:23.217997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:49.038621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218340+0000","last_change":"2024-09-21T07:26:14.756162+0000","last_active":"2024-09-21T07:26:23.218340+0000","last_peered":"2024-09-21T07:26:23.218340+0000","last_clean":"2024-09-21T07:26:23.218340+0000","last_became_active":"2024-09-21T07:26:14.755821+0000","last_became_peered":"2024-09-21T07:26:14.755821+0000","last_unstale":"2024-09-21T07:26:23.218340+0000","last_undegraded":"2024-09-21T07:26:23.218340+0000","last_fullsized":"2024-09-21T07:26:23.218340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217766+0000","last_change":"2024-09-21T07:26:19.782715+0000","last_active":"2024-09-21T07:26:23.217766+0000","last_peered":"2024-09-21T07:26:23.217766+0000","last_clean":"2024-09-21T07:26:23.217766+0000","last_became_active":"2024-09-21T07:26:19.782285+0000","last_became_peered":"2024-09-21T07:26:19.782285+0000","last_unstale":"2024-09-21T07:26:23.217766+0000","last_undegraded":"2024-09-21T07:26:23.217766+0000","last_fullsized":"2024-09-21T07:26:23.217766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756175+0000","last_change":"2024-09-21T07:26:15.759805+0000","last_active":"2024-09-21T07:26:22.756175+0000","last_peered":"2024-09-21T07:26:22.756175+0000","last_clean":"2024-09-21T07:26:22.756175+0000","last_became_active":"2024-09-21T07:26:15.759487+0000","last_became_peered":"2024-09-21T07:26:15.759487+0000","last_unstale":"2024-09-21T07:26:22.756175+0000","last_undegraded":"2024-09-21T07:26:22.756175+0000","last_fullsized":"2024-09-21T07:26:22.756175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900206+0000","last_change":"2024-09-21T07:26:14.753655+0000","last_active":"2024-09-21T07:26:22.900206+0000","last_peered":"2024-09-21T07:26:22.900206+0000","last_clean":"2024-09-21T07:26:22.900206+0000","last_became_active":"2024-09-21T07:26:14.753266+0000","last_became_peered":"2024-09-21T07:26:14.753266+0000","last_unstale":"2024-09-21T07:26:22.900206+0000","last_undegraded":"2024-09-21T07:26:22.900206+0000","last_fullsized":"2024-09-21T07:26:22.900206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:14.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218754+0000","last_change":"2024-09-21T07:26:19.786292+0000","last_active":"2024-09-21T07:26:23.218754+0000","last_peered":"2024-09-21T07:26:23.218754+0000","last_clean":"2024-09-21T07:26:23.218754+0000","last_became_active":"2024-09-21T07:26:19.783800+0000","last_became_peered":"2024-09-21T07:26:19.783800+0000","last_unstale":"2024-09-21T07:26:23.218754+0000","last_undegraded":"2024-09-21T07:26:23.218754+0000","last_fullsized":"2024-09-21T07:26:23.218754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:52.992488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901112+0000","last_change":"2024-09-21T07:26:15.766957+0000","last_active":"2024-09-21T07:26:22.901112+0000","last_peered":"2024-09-21T07:26:22.901112+0000","last_clean":"2024-09-21T07:26:22.901112+0000","last_became_active":"2024-09-21T07:26:15.766640+0000","last_became_peered":"2024-09-21T07:26:15.766640+0000","last_unstale":"2024-09-21T07:26:22.901112+0000","last_undegraded":"2024-09-21T07:26:22.901112+0000","last_fullsized":"2024-09-21T07:26:22.901112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:23.200622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791558+0000","last_change":"2024-09-21T07:26:14.756852+0000","last_active":"2024-09-21T07:26:21.791558+0000","last_peered":"2024-09-21T07:26:21.791558+0000","last_clean":"2024-09-21T07:26:21.791558+0000","last_became_active":"2024-09-21T07:26:14.756512+0000","last_became_peered":"2024-09-21T07:26:14.756512+0000","last_unstale":"2024-09-21T07:26:21.791558+0000","last_undegraded":"2024-09-21T07:26:21.791558+0000","last_fullsized":"2024-09-21T07:26:21.791558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217003+0000","last_change":"2024-09-21T07:26:20.205385+0000","last_active":"2024-09-21T07:26:23.217003+0000","last_peered":"2024-09-21T07:26:23.217003+0000","last_clean":"2024-09-21T07:26:23.217003+0000","last_became_active":"2024-09-21T07:26:20.205016+0000","last_became_peered":"2024-09-21T07:26:20.205016+0000","last_unstale":"2024-09-21T07:26:23.217003+0000","last_undegraded":"2024-09-21T07:26:23.217003+0000","last_fullsized":"2024-09-21T07:26:23.217003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.612156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755846+0000","last_change":"2024-09-21T07:26:20.202075+0000","last_active":"2024-09-21T07:26:22.755846+0000","last_peered":"2024-09-21T07:26:22.755846+0000","last_clean":"2024-09-21T07:26:22.755846+0000","last_became_active":"2024-09-21T07:26:20.201632+0000","last_became_peered":"2024-09-21T07:26:20.201632+0000","last_unstale":"2024-09-21T07:26:22.755846+0000","last_undegraded":"2024-09-21T07:26:22.755846+0000","last_fullsized":"2024-09-21T07:26:22.755846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216493+0000","last_change":"2024-09-21T07:26:15.762928+0000","last_active":"2024-09-21T07:26:23.216493+0000","last_peered":"2024-09-21T07:26:23.216493+0000","last_clean":"2024-09-21T07:26:23.216493+0000","last_became_active":"2024-09-21T07:26:15.762291+0000","last_became_peered":"2024-09-21T07:26:15.762291+0000","last_unstale":"2024-09-21T07:26:23.216493+0000","last_undegraded":"2024-09-21T07:26:23.216493+0000","last_fullsized":"2024-09-21T07:26:23.216493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900637+0000","last_change":"2024-09-21T07:26:14.755764+0000","last_active":"2024-09-21T07:26:22.900637+0000","last_peered":"2024-09-21T07:26:22.900637+0000","last_clean":"2024-09-21T07:26:22.900637+0000","last_became_active":"2024-09-21T07:26:14.755397+0000","last_became_peered":"2024-09-21T07:26:14.755397+0000","last_unstale":"2024-09-21T07:26:22.900637+0000","last_undegraded":"2024-09-21T07:26:22.900637+0000","last_fullsized":"2024-09-21T07:26:22.900637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:32.946211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791301+0000","last_change":"2024-09-21T07:26:19.789958+0000","last_active":"2024-09-21T07:26:21.791301+0000","last_peered":"2024-09-21T07:26:21.791301+0000","last_clean":"2024-09-21T07:26:21.791301+0000","last_became_active":"2024-09-21T07:26:19.789515+0000","last_became_peered":"2024-09-21T07:26:19.789515+0000","last_unstale":"2024-09-21T07:26:21.791301+0000","last_undegraded":"2024-09-21T07:26:21.791301+0000","last_fullsized":"2024-09-21T07:26:21.791301+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218685+0000","last_change":"2024-09-21T07:26:15.760941+0000","last_active":"2024-09-21T07:26:23.218685+0000","last_peered":"2024-09-21T07:26:23.218685+0000","last_clean":"2024-09-21T07:26:23.218685+0000","last_became_active":"2024-09-21T07:26:15.760612+0000","last_became_peered":"2024-09-21T07:26:15.760612+0000","last_unstale":"2024-09-21T07:26:23.218685+0000","last_undegraded":"2024-09-21T07:26:23.218685+0000","last_fullsized":"2024-09-21T07:26:23.218685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:23.610491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216574+0000","last_change":"2024-09-21T07:26:14.744602+0000","last_active":"2024-09-21T07:26:23.216574+0000","last_peered":"2024-09-21T07:26:23.216574+0000","last_clean":"2024-09-21T07:26:23.216574+0000","last_became_active":"2024-09-21T07:26:14.744284+0000","last_became_peered":"2024-09-21T07:26:14.744284+0000","last_unstale":"2024-09-21T07:26:23.216574+0000","last_undegraded":"2024-09-21T07:26:23.216574+0000","last_fullsized":"2024-09-21T07:26:23.216574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:39.642931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790820+0000","last_change":"2024-09-21T07:26:14.750830+0000","last_active":"2024-09-21T07:26:21.790820+0000","last_peered":"2024-09-21T07:26:21.790820+0000","last_clean":"2024-09-21T07:26:21.790820+0000","last_became_active":"2024-09-21T07:26:14.749815+0000","last_became_peered":"2024-09-21T07:26:14.749815+0000","last_unstale":"2024-09-21T07:26:21.790820+0000","last_undegraded":"2024-09-21T07:26:21.790820+0000","last_fullsized":"2024-09-21T07:26:21.790820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756340+0000","last_change":"2024-09-21T07:26:15.773671+0000","last_active":"2024-09-21T07:26:22.756340+0000","last_peered":"2024-09-21T07:26:22.756340+0000","last_clean":"2024-09-21T07:26:22.756340+0000","last_became_active":"2024-09-21T07:26:15.773380+0000","last_became_peered":"2024-09-21T07:26:15.773380+0000","last_unstale":"2024-09-21T07:26:22.756340+0000","last_undegraded":"2024-09-21T07:26:22.756340+0000","last_fullsized":"2024-09-21T07:26:22.756340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789217+0000","last_change":"2024-09-21T07:26:19.785278+0000","last_active":"2024-09-21T07:26:21.789217+0000","last_peered":"2024-09-21T07:26:21.789217+0000","last_clean":"2024-09-21T07:26:21.789217+0000","last_became_active":"2024-09-21T07:26:19.784142+0000","last_became_peered":"2024-09-21T07:26:19.784142+0000","last_unstale":"2024-09-21T07:26:21.789217+0000","last_undegraded":"2024-09-21T07:26:21.789217+0000","last_fullsized":"2024-09-21T07:26:21.789217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.144434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217614+0000","last_change":"2024-09-21T07:26:14.746012+0000","last_active":"2024-09-21T07:26:23.217614+0000","last_peered":"2024-09-21T07:26:23.217614+0000","last_clean":"2024-09-21T07:26:23.217614+0000","last_became_active":"2024-09-21T07:26:14.745668+0000","last_became_peered":"2024-09-21T07:26:14.745668+0000","last_unstale":"2024-09-21T07:26:23.217614+0000","last_undegraded":"2024-09-21T07:26:23.217614+0000","last_fullsized":"2024-09-21T07:26:23.217614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:03.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217510+0000","last_change":"2024-09-21T07:26:15.764689+0000","last_active":"2024-09-21T07:26:23.217510+0000","last_peered":"2024-09-21T07:26:23.217510+0000","last_clean":"2024-09-21T07:26:23.217510+0000","last_became_active":"2024-09-21T07:26:15.763516+0000","last_became_peered":"2024-09-21T07:26:15.763516+0000","last_unstale":"2024-09-21T07:26:23.217510+0000","last_undegraded":"2024-09-21T07:26:23.217510+0000","last_fullsized":"2024-09-21T07:26:23.217510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.556036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217467+0000","last_change":"2024-09-21T07:26:19.784787+0000","last_active":"2024-09-21T07:26:23.217467+0000","last_peered":"2024-09-21T07:26:23.217467+0000","last_clean":"2024-09-21T07:26:23.217467+0000","last_became_active":"2024-09-21T07:26:19.784262+0000","last_became_peered":"2024-09-21T07:26:19.784262+0000","last_unstale":"2024-09-21T07:26:23.217467+0000","last_undegraded":"2024-09-21T07:26:23.217467+0000","last_fullsized":"2024-09-21T07:26:23.217467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:47.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790131+0000","last_change":"2024-09-21T07:26:15.762178+0000","last_active":"2024-09-21T07:26:21.790131+0000","last_peered":"2024-09-21T07:26:21.790131+0000","last_clean":"2024-09-21T07:26:21.790131+0000","last_became_active":"2024-09-21T07:26:15.761535+0000","last_became_peered":"2024-09-21T07:26:15.761535+0000","last_unstale":"2024-09-21T07:26:21.790131+0000","last_undegraded":"2024-09-21T07:26:21.790131+0000","last_fullsized":"2024-09-21T07:26:21.790131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789216+0000","last_change":"2024-09-21T07:26:15.760862+0000","last_active":"2024-09-21T07:26:21.789216+0000","last_peered":"2024-09-21T07:26:21.789216+0000","last_clean":"2024-09-21T07:26:21.789216+0000","last_became_active":"2024-09-21T07:26:15.760547+0000","last_became_peered":"2024-09-21T07:26:15.760547+0000","last_unstale":"2024-09-21T07:26:21.789216+0000","last_undegraded":"2024-09-21T07:26:21.789216+0000","last_fullsized":"2024-09-21T07:26:21.789216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:49.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900702+0000","last_change":"2024-09-21T07:26:15.773293+0000","last_active":"2024-09-21T07:26:22.900702+0000","last_peered":"2024-09-21T07:26:22.900702+0000","last_clean":"2024-09-21T07:26:22.900702+0000","last_became_active":"2024-09-21T07:26:15.772904+0000","last_became_peered":"2024-09-21T07:26:15.772904+0000","last_unstale":"2024-09-21T07:26:22.900702+0000","last_undegraded":"2024-09-21T07:26:22.900702+0000","last_fullsized":"2024-09-21T07:26:22.900702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:56.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756234+0000","last_change":"2024-09-21T07:26:15.764081+0000","last_active":"2024-09-21T07:26:22.756234+0000","last_peered":"2024-09-21T07:26:22.756234+0000","last_clean":"2024-09-21T07:26:22.756234+0000","last_became_active":"2024-09-21T07:26:15.763052+0000","last_became_peered":"2024-09-21T07:26:15.763052+0000","last_unstale":"2024-09-21T07:26:22.756234+0000","last_undegraded":"2024-09-21T07:26:22.756234+0000","last_fullsized":"2024-09-21T07:26:22.756234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218342+0000","last_change":"2024-09-21T07:26:15.766994+0000","last_active":"2024-09-21T07:26:23.218342+0000","last_peered":"2024-09-21T07:26:23.218342+0000","last_clean":"2024-09-21T07:26:23.218342+0000","last_became_active":"2024-09-21T07:26:15.766623+0000","last_became_peered":"2024-09-21T07:26:15.766623+0000","last_unstale":"2024-09-21T07:26:23.218342+0000","last_undegraded":"2024-09-21T07:26:23.218342+0000","last_fullsized":"2024-09-21T07:26:23.218342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:59.716664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218240+0000","last_change":"2024-09-21T07:26:15.758623+0000","last_active":"2024-09-21T07:26:23.218240+0000","last_peered":"2024-09-21T07:26:23.218240+0000","last_clean":"2024-09-21T07:26:23.218240+0000","last_became_active":"2024-09-21T07:26:15.758309+0000","last_became_peered":"2024-09-21T07:26:15.758309+0000","last_unstale":"2024-09-21T07:26:23.218240+0000","last_undegraded":"2024-09-21T07:26:23.218240+0000","last_fullsized":"2024-09-21T07:26:23.218240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792006+0000","last_change":"2024-09-21T07:26:15.770009+0000","last_active":"2024-09-21T07:26:21.792006+0000","last_peered":"2024-09-21T07:26:21.792006+0000","last_clean":"2024-09-21T07:26:21.792006+0000","last_became_active":"2024-09-21T07:26:15.769549+0000","last_became_peered":"2024-09-21T07:26:15.769549+0000","last_unstale":"2024-09-21T07:26:21.792006+0000","last_undegraded":"2024-09-21T07:26:21.792006+0000","last_fullsized":"2024-09-21T07:26:21.792006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218223+0000","last_change":"2024-09-21T07:26:15.765956+0000","last_active":"2024-09-21T07:26:23.218223+0000","last_peered":"2024-09-21T07:26:23.218223+0000","last_clean":"2024-09-21T07:26:23.218223+0000","last_became_active":"2024-09-21T07:26:15.765561+0000","last_became_peered":"2024-09-21T07:26:15.765561+0000","last_unstale":"2024-09-21T07:26:23.218223+0000","last_undegraded":"2024-09-21T07:26:23.218223+0000","last_fullsized":"2024-09-21T07:26:23.218223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:35.944784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900539+0000","last_change":"2024-09-21T07:26:15.766091+0000","last_active":"2024-09-21T07:26:22.900539+0000","last_peered":"2024-09-21T07:26:22.900539+0000","last_clean":"2024-09-21T07:26:22.900539+0000","last_became_active":"2024-09-21T07:26:15.765771+0000","last_became_peered":"2024-09-21T07:26:15.765771+0000","last_unstale":"2024-09-21T07:26:22.900539+0000","last_undegraded":"2024-09-21T07:26:22.900539+0000","last_fullsized":"2024-09-21T07:26:22.900539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:15.463622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217240+0000","last_change":"2024-09-21T07:26:15.771905+0000","last_active":"2024-09-21T07:26:23.217240+0000","last_peered":"2024-09-21T07:26:23.217240+0000","last_clean":"2024-09-21T07:26:23.217240+0000","last_became_active":"2024-09-21T07:26:15.771525+0000","last_became_peered":"2024-09-21T07:26:15.771525+0000","last_unstale":"2024-09-21T07:26:23.217240+0000","last_undegraded":"2024-09-21T07:26:23.217240+0000","last_fullsized":"2024-09-21T07:26:23.217240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.988043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900725+0000","last_change":"2024-09-21T07:26:15.775865+0000","last_active":"2024-09-21T07:26:22.900725+0000","last_peered":"2024-09-21T07:26:22.900725+0000","last_clean":"2024-09-21T07:26:22.900725+0000","last_became_active":"2024-09-21T07:26:15.773173+0000","last_became_peered":"2024-09-21T07:26:15.773173+0000","last_unstale":"2024-09-21T07:26:22.900725+0000","last_undegraded":"2024-09-21T07:26:22.900725+0000","last_fullsized":"2024-09-21T07:26:22.900725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:42.898368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756041+0000","last_change":"2024-09-21T07:26:15.766057+0000","last_active":"2024-09-21T07:26:22.756041+0000","last_peered":"2024-09-21T07:26:22.756041+0000","last_clean":"2024-09-21T07:26:22.756041+0000","last_became_active":"2024-09-21T07:26:15.765714+0000","last_became_peered":"2024-09-21T07:26:15.765714+0000","last_unstale":"2024-09-21T07:26:22.756041+0000","last_undegraded":"2024-09-21T07:26:22.756041+0000","last_fullsized":"2024-09-21T07:26:22.756041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790549+0000","last_change":"2024-09-21T07:26:15.774046+0000","last_active":"2024-09-21T07:26:21.790549+0000","last_peered":"2024-09-21T07:26:21.790549+0000","last_clean":"2024-09-21T07:26:21.790549+0000","last_became_active":"2024-09-21T07:26:15.773709+0000","last_became_peered":"2024-09-21T07:26:15.773709+0000","last_unstale":"2024-09-21T07:26:21.790549+0000","last_undegraded":"2024-09-21T07:26:21.790549+0000","last_fullsized":"2024-09-21T07:26:21.790549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791898+0000","last_change":"2024-09-21T07:26:15.760349+0000","last_active":"2024-09-21T07:26:21.791898+0000","last_peered":"2024-09-21T07:26:21.791898+0000","last_clean":"2024-09-21T07:26:21.791898+0000","last_became_active":"2024-09-21T07:26:15.759759+0000","last_became_peered":"2024-09-21T07:26:15.759759+0000","last_unstale":"2024-09-21T07:26:21.791898+0000","last_undegraded":"2024-09-21T07:26:21.791898+0000","last_fullsized":"2024-09-21T07:26:21.791898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:33.433481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789376+0000","last_change":"2024-09-21T07:26:15.767539+0000","last_active":"2024-09-21T07:26:21.789376+0000","last_peered":"2024-09-21T07:26:21.789376+0000","last_clean":"2024-09-21T07:26:21.789376+0000","last_became_active":"2024-09-21T07:26:15.767174+0000","last_became_peered":"2024-09-21T07:26:15.767174+0000","last_unstale":"2024-09-21T07:26:21.789376+0000","last_undegraded":"2024-09-21T07:26:21.789376+0000","last_fullsized":"2024-09-21T07:26:21.789376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:08.568390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901369+0000","last_change":"2024-09-21T07:26:15.767788+0000","last_active":"2024-09-21T07:26:22.901369+0000","last_peered":"2024-09-21T07:26:22.901369+0000","last_clean":"2024-09-21T07:26:22.901369+0000","last_became_active":"2024-09-21T07:26:15.767459+0000","last_became_peered":"2024-09-21T07:26:15.767459+0000","last_unstale":"2024-09-21T07:26:22.901369+0000","last_undegraded":"2024-09-21T07:26:22.901369+0000","last_fullsized":"2024-09-21T07:26:22.901369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.318947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900393+0000","last_change":"2024-09-21T07:26:15.758591+0000","last_active":"2024-09-21T07:26:22.900393+0000","last_peered":"2024-09-21T07:26:22.900393+0000","last_clean":"2024-09-21T07:26:22.900393+0000","last_became_active":"2024-09-21T07:26:15.758248+0000","last_became_peered":"2024-09-21T07:26:15.758248+0000","last_unstale":"2024-09-21T07:26:22.900393+0000","last_undegraded":"2024-09-21T07:26:22.900393+0000","last_fullsized":"2024-09-21T07:26:22.900393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:36.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216601+0000","last_change":"2024-09-21T07:26:15.763665+0000","last_active":"2024-09-21T07:26:23.216601+0000","last_peered":"2024-09-21T07:26:23.216601+0000","last_clean":"2024-09-21T07:26:23.216601+0000","last_became_active":"2024-09-21T07:26:15.763348+0000","last_became_peered":"2024-09-21T07:26:15.763348+0000","last_unstale":"2024-09-21T07:26:23.216601+0000","last_undegraded":"2024-09-21T07:26:23.216601+0000","last_fullsized":"2024-09-21T07:26:23.216601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755781+0000","last_change":"2024-09-21T07:26:15.757427+0000","last_active":"2024-09-21T07:26:22.755781+0000","last_peered":"2024-09-21T07:26:22.755781+0000","last_clean":"2024-09-21T07:26:22.755781+0000","last_became_active":"2024-09-21T07:26:15.757078+0000","last_became_peered":"2024-09-21T07:26:15.757078+0000","last_unstale":"2024-09-21T07:26:22.755781+0000","last_undegraded":"2024-09-21T07:26:22.755781+0000","last_fullsized":"2024-09-21T07:26:22.755781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:46.647829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:15.767064+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:15.766690+0000","last_became_peered":"2024-09-21T07:26:15.766690+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:07.860716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217825+0000","last_change":"2024-09-21T07:26:15.775975+0000","last_active":"2024-09-21T07:26:23.217825+0000","last_peered":"2024-09-21T07:26:23.217825+0000","last_clean":"2024-09-21T07:26:23.217825+0000","last_became_active":"2024-09-21T07:26:15.775526+0000","last_became_peered":"2024-09-21T07:26:15.775526+0000","last_unstale":"2024-09-21T07:26:23.217825+0000","last_undegraded":"2024-09-21T07:26:23.217825+0000","last_fullsized":"2024-09-21T07:26:23.217825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.840733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791070+0000","last_change":"2024-09-21T07:26:15.776942+0000","last_active":"2024-09-21T07:26:21.791070+0000","last_peered":"2024-09-21T07:26:21.791070+0000","last_clean":"2024-09-21T07:26:21.791070+0000","last_became_active":"2024-09-21T07:26:15.776632+0000","last_became_peered":"2024-09-21T07:26:15.776632+0000","last_unstale":"2024-09-21T07:26:21.791070+0000","last_undegraded":"2024-09-21T07:26:21.791070+0000","last_fullsized":"2024-09-21T07:26:21.791070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791878+0000","last_change":"2024-09-21T07:26:15.766327+0000","last_active":"2024-09-21T07:26:21.791878+0000","last_peered":"2024-09-21T07:26:21.791878+0000","last_clean":"2024-09-21T07:26:21.791878+0000","last_became_active":"2024-09-21T07:26:15.765981+0000","last_became_peered":"2024-09-21T07:26:15.765981+0000","last_unstale":"2024-09-21T07:26:21.791878+0000","last_undegraded":"2024-09-21T07:26:21.791878+0000","last_fullsized":"2024-09-21T07:26:21.791878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791919+0000","last_change":"2024-09-21T07:26:15.777412+0000","last_active":"2024-09-21T07:26:21.791919+0000","last_peered":"2024-09-21T07:26:21.791919+0000","last_clean":"2024-09-21T07:26:21.791919+0000","last_became_active":"2024-09-21T07:26:15.776925+0000","last_became_peered":"2024-09-21T07:26:15.776925+0000","last_unstale":"2024-09-21T07:26:21.791919+0000","last_undegraded":"2024-09-21T07:26:21.791919+0000","last_fullsized":"2024-09-21T07:26:21.791919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:38.270679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218556+0000","last_change":"2024-09-21T07:26:15.777075+0000","last_active":"2024-09-21T07:26:23.218556+0000","last_peered":"2024-09-21T07:26:23.218556+0000","last_clean":"2024-09-21T07:26:23.218556+0000","last_became_active":"2024-09-21T07:26:15.776611+0000","last_became_peered":"2024-09-21T07:26:15.776611+0000","last_unstale":"2024-09-21T07:26:23.218556+0000","last_undegraded":"2024-09-21T07:26:23.218556+0000","last_fullsized":"2024-09-21T07:26:23.218556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:38.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790866+0000","last_change":"2024-09-21T07:26:15.762408+0000","last_active":"2024-09-21T07:26:21.790866+0000","last_peered":"2024-09-21T07:26:21.790866+0000","last_clean":"2024-09-21T07:26:21.790866+0000","last_became_active":"2024-09-21T07:26:15.762014+0000","last_became_peered":"2024-09-21T07:26:15.762014+0000","last_unstale":"2024-09-21T07:26:21.790866+0000","last_undegraded":"2024-09-21T07:26:21.790866+0000","last_fullsized":"2024-09-21T07:26:21.790866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:09.186226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900571+0000","last_change":"2024-09-21T07:26:15.759759+0000","last_active":"2024-09-21T07:26:22.900571+0000","last_peered":"2024-09-21T07:26:22.900571+0000","last_clean":"2024-09-21T07:26:22.900571+0000","last_became_active":"2024-09-21T07:26:15.758871+0000","last_became_peered":"2024-09-21T07:26:15.758871+0000","last_unstale":"2024-09-21T07:26:22.900571+0000","last_undegraded":"2024-09-21T07:26:22.900571+0000","last_fullsized":"2024-09-21T07:26:22.900571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:23.666201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791461+0000","last_change":"2024-09-21T07:26:15.768245+0000","last_active":"2024-09-21T07:26:21.791461+0000","last_peered":"2024-09-21T07:26:21.791461+0000","last_clean":"2024-09-21T07:26:21.791461+0000","last_became_active":"2024-09-21T07:26:15.767901+0000","last_became_peered":"2024-09-21T07:26:15.767901+0000","last_unstale":"2024-09-21T07:26:21.791461+0000","last_undegraded":"2024-09-21T07:26:21.791461+0000","last_fullsized":"2024-09-21T07:26:21.791461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:52.823399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900922+0000","last_change":"2024-09-21T07:26:15.772040+0000","last_active":"2024-09-21T07:26:22.900922+0000","last_peered":"2024-09-21T07:26:22.900922+0000","last_clean":"2024-09-21T07:26:22.900922+0000","last_became_active":"2024-09-21T07:26:15.771194+0000","last_became_peered":"2024-09-21T07:26:15.771194+0000","last_unstale":"2024-09-21T07:26:22.900922+0000","last_undegraded":"2024-09-21T07:26:22.900922+0000","last_fullsized":"2024-09-21T07:26:22.900922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:35.969051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755834+0000","last_change":"2024-09-21T07:26:15.752787+0000","last_active":"2024-09-21T07:26:22.755834+0000","last_peered":"2024-09-21T07:26:22.755834+0000","last_clean":"2024-09-21T07:26:22.755834+0000","last_became_active":"2024-09-21T07:26:15.752457+0000","last_became_peered":"2024-09-21T07:26:15.752457+0000","last_unstale":"2024-09-21T07:26:22.755834+0000","last_undegraded":"2024-09-21T07:26:22.755834+0000","last_fullsized":"2024-09-21T07:26:22.755834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217328+0000","last_change":"2024-09-21T07:26:15.754185+0000","last_active":"2024-09-21T07:26:23.217328+0000","last_peered":"2024-09-21T07:26:23.217328+0000","last_clean":"2024-09-21T07:26:23.217328+0000","last_became_active":"2024-09-21T07:26:15.753790+0000","last_became_peered":"2024-09-21T07:26:15.753790+0000","last_unstale":"2024-09-21T07:26:23.217328+0000","last_undegraded":"2024-09-21T07:26:23.217328+0000","last_fullsized":"2024-09-21T07:26:23.217328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.769408+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.767688+0000","last_became_peered":"2024-09-21T07:26:15.767688+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:46.925801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":704845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":113985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":114021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":113895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":114055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":888832,"data_stored":704901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:26:41.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:26:41.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 -- 172.21.15.105:0/60540605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3240b6440 msgr2=0x7fc3240cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 --2- 172.21.15.105:0/60540605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3240b6440 0x7fc3240cf050 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc318005fd0 tx=0x7fc31802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 -- 172.21.15.105:0/60540605 shutdown_connections 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 --2- 172.21.15.105:0/60540605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3240b6440 0x7fc3240cf050 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 --2- 172.21.15.105:0/60540605 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc32408c3c0 0x7fc32406f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 -- 172.21.15.105:0/60540605 >> 172.21.15.105:0/60540605 conn(0x7fc32408bfb0 msgr2=0x7fc3240d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 -- 172.21.15.105:0/60540605 shutdown_connections 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.824+0000 7fc32aab4640 1 -- 172.21.15.105:0/60540605 wait complete. 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.826+0000 7fc32aab4640 1 Processor -- start 2024-09-21T07:26:41.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.826+0000 7fc32aab4640 1 -- start start 2024-09-21T07:26:41.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3240b6440 0x7fc324160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3240b6850 0x7fc3241e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b3ad0 con 0x7fc32408c3c0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240b3950 con 0x7fc3240b6440 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc32aab4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc324162c70 con 0x7fc3240b6850 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60718/0 (socket says 172.21.15.105:60718) 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 -- 172.21.15.105:0/1742848093 learned_addr learned my addr 172.21.15.105:0/1742848093 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc323fff640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3240b6440 0x7fc324160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc329051640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3240b6850 0x7fc3241e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 -- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3240b6850 msgr2=0x7fc3241e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3240b6850 0x7fc3241e8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 -- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3240b6440 msgr2=0x7fc324160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3240b6440 0x7fc324160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.827+0000 7fc328850640 1 -- 172.21.15.105:0/1742848093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3240b75f0 con 0x7fc32408c3c0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc328850640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c970 tx=0x7fc31400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:41.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314007bb0 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc314007e40 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3240b7860 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314005280 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.828+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3240b7e20 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.829+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32406f7f0 con 0x7fc32408c3c0 2024-09-21T07:26:41.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.830+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc314005420 con 0x7fc32408c3c0 2024-09-21T07:26:41.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.830+0000 7fc3217fa640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:41.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.830+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc31409b4b0 con 0x7fc32408c3c0 2024-09-21T07:26:41.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.830+0000 7fc323fff640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:41.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.831+0000 7fc323fff640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 0x7fc31007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc318033000 tx=0x7fc318004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:41.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:41.835+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc314014030 con 0x7fc32408c3c0 2024-09-21T07:26:42.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.009+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3240b2e10 con 0x7fc310078b90 2024-09-21T07:26:42.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.015+0000 7fc3217fa640 1 -- 172.21.15.105:0/1742848093 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332523 (secure 0 0 0) 0x7fc3240b2e10 con 0x7fc310078b90 2024-09-21T07:26:42.020 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:42.022 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:26:42.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 msgr2=0x7fc31007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 0x7fc31007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc318033000 tx=0x7fc318004730 comp rx=0 tx=0).stop 2024-09-21T07:26:42.023 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 msgr2=0x7fc324159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c970 tx=0x7fc31400ce30 comp rx=0 tx=0).stop 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 shutdown_connections 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc310078b90 0x7fc31007b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3240b6440 0x7fc324160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3240b6850 0x7fc3241e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 --2- 172.21.15.105:0/1742848093 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc32408c3c0 0x7fc324159d10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 >> 172.21.15.105:0/1742848093 conn(0x7fc32408bfb0 msgr2=0x7fc32406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 shutdown_connections 2024-09-21T07:26:42.024 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.018+0000 7fc32aab4640 1 -- 172.21.15.105:0/1742848093 wait complete. 2024-09-21T07:26:42.031 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-21T07:26:40.771809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3514368,"data_stored":2101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003251"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755382+0000","last_change":"2024-09-21T07:26:14.746008+0000","last_active":"2024-09-21T07:26:22.755382+0000","last_peered":"2024-09-21T07:26:22.755382+0000","last_clean":"2024-09-21T07:26:22.755382+0000","last_became_active":"2024-09-21T07:26:14.745685+0000","last_became_peered":"2024-09-21T07:26:14.745685+0000","last_unstale":"2024-09-21T07:26:22.755382+0000","last_undegraded":"2024-09-21T07:26:22.755382+0000","last_fullsized":"2024-09-21T07:26:22.755382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:37:02.797953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217709+0000","last_change":"2024-09-21T07:26:15.767737+0000","last_active":"2024-09-21T07:26:23.217709+0000","last_peered":"2024-09-21T07:26:23.217709+0000","last_clean":"2024-09-21T07:26:23.217709+0000","last_became_active":"2024-09-21T07:26:15.767112+0000","last_became_peered":"2024-09-21T07:26:15.767112+0000","last_unstale":"2024-09-21T07:26:23.217709+0000","last_undegraded":"2024-09-21T07:26:23.217709+0000","last_fullsized":"2024-09-21T07:26:23.217709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:20.781454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217413+0000","last_change":"2024-09-21T07:26:20.205421+0000","last_active":"2024-09-21T07:26:23.217413+0000","last_peered":"2024-09-21T07:26:23.217413+0000","last_clean":"2024-09-21T07:26:23.217413+0000","last_became_active":"2024-09-21T07:26:20.204943+0000","last_became_peered":"2024-09-21T07:26:20.204943+0000","last_unstale":"2024-09-21T07:26:23.217413+0000","last_undegraded":"2024-09-21T07:26:23.217413+0000","last_fullsized":"2024-09-21T07:26:23.217413+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:55.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:14.738042+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:14.737714+0000","last_became_peered":"2024-09-21T07:26:14.737714+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:43.999074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756065+0000","last_change":"2024-09-21T07:26:15.760507+0000","last_active":"2024-09-21T07:26:22.756065+0000","last_peered":"2024-09-21T07:26:22.756065+0000","last_clean":"2024-09-21T07:26:22.756065+0000","last_became_active":"2024-09-21T07:26:15.759859+0000","last_became_peered":"2024-09-21T07:26:15.759859+0000","last_unstale":"2024-09-21T07:26:22.756065+0000","last_undegraded":"2024-09-21T07:26:22.756065+0000","last_fullsized":"2024-09-21T07:26:22.756065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217096+0000","last_change":"2024-09-21T07:26:19.783194+0000","last_active":"2024-09-21T07:26:23.217096+0000","last_peered":"2024-09-21T07:26:23.217096+0000","last_clean":"2024-09-21T07:26:23.217096+0000","last_became_active":"2024-09-21T07:26:19.782625+0000","last_became_peered":"2024-09-21T07:26:19.782625+0000","last_unstale":"2024-09-21T07:26:23.217096+0000","last_undegraded":"2024-09-21T07:26:23.217096+0000","last_fullsized":"2024-09-21T07:26:23.217096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791768+0000","last_change":"2024-09-21T07:26:14.748310+0000","last_active":"2024-09-21T07:26:21.791768+0000","last_peered":"2024-09-21T07:26:21.791768+0000","last_clean":"2024-09-21T07:26:21.791768+0000","last_became_active":"2024-09-21T07:26:14.747943+0000","last_became_peered":"2024-09-21T07:26:14.747943+0000","last_unstale":"2024-09-21T07:26:21.791768+0000","last_undegraded":"2024-09-21T07:26:21.791768+0000","last_fullsized":"2024-09-21T07:26:21.791768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:12.833502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901017+0000","last_change":"2024-09-21T07:26:15.772109+0000","last_active":"2024-09-21T07:26:22.901017+0000","last_peered":"2024-09-21T07:26:22.901017+0000","last_clean":"2024-09-21T07:26:22.901017+0000","last_became_active":"2024-09-21T07:26:15.771762+0000","last_became_peered":"2024-09-21T07:26:15.771762+0000","last_unstale":"2024-09-21T07:26:22.901017+0000","last_undegraded":"2024-09-21T07:26:22.901017+0000","last_fullsized":"2024-09-21T07:26:22.901017+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:14.024694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217587+0000","last_change":"2024-09-21T07:26:19.782636+0000","last_active":"2024-09-21T07:26:23.217587+0000","last_peered":"2024-09-21T07:26:23.217587+0000","last_clean":"2024-09-21T07:26:23.217587+0000","last_became_active":"2024-09-21T07:26:19.782148+0000","last_became_peered":"2024-09-21T07:26:19.782148+0000","last_unstale":"2024-09-21T07:26:23.217587+0000","last_undegraded":"2024-09-21T07:26:23.217587+0000","last_fullsized":"2024-09-21T07:26:23.217587+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:34.102740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791003+0000","last_change":"2024-09-21T07:26:14.756864+0000","last_active":"2024-09-21T07:26:21.791003+0000","last_peered":"2024-09-21T07:26:21.791003+0000","last_clean":"2024-09-21T07:26:21.791003+0000","last_became_active":"2024-09-21T07:26:14.756519+0000","last_became_peered":"2024-09-21T07:26:14.756519+0000","last_unstale":"2024-09-21T07:26:21.791003+0000","last_undegraded":"2024-09-21T07:26:21.791003+0000","last_fullsized":"2024-09-21T07:26:21.791003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:48.306611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901081+0000","last_change":"2024-09-21T07:26:15.769890+0000","last_active":"2024-09-21T07:26:22.901081+0000","last_peered":"2024-09-21T07:26:22.901081+0000","last_clean":"2024-09-21T07:26:22.901081+0000","last_became_active":"2024-09-21T07:26:15.769124+0000","last_became_peered":"2024-09-21T07:26:15.769124+0000","last_unstale":"2024-09-21T07:26:22.901081+0000","last_undegraded":"2024-09-21T07:26:22.901081+0000","last_fullsized":"2024-09-21T07:26:22.901081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:55.122043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217329+0000","last_change":"2024-09-21T07:26:19.788443+0000","last_active":"2024-09-21T07:26:23.217329+0000","last_peered":"2024-09-21T07:26:23.217329+0000","last_clean":"2024-09-21T07:26:23.217329+0000","last_became_active":"2024-09-21T07:26:19.787922+0000","last_became_peered":"2024-09-21T07:26:19.787922+0000","last_unstale":"2024-09-21T07:26:23.217329+0000","last_undegraded":"2024-09-21T07:26:23.217329+0000","last_fullsized":"2024-09-21T07:26:23.217329+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790557+0000","last_change":"2024-09-21T07:26:15.774537+0000","last_active":"2024-09-21T07:26:21.790557+0000","last_peered":"2024-09-21T07:26:21.790557+0000","last_clean":"2024-09-21T07:26:21.790557+0000","last_became_active":"2024-09-21T07:26:15.774215+0000","last_became_peered":"2024-09-21T07:26:15.774215+0000","last_unstale":"2024-09-21T07:26:21.790557+0000","last_undegraded":"2024-09-21T07:26:21.790557+0000","last_fullsized":"2024-09-21T07:26:21.790557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791990+0000","last_change":"2024-09-21T07:26:14.752255+0000","last_active":"2024-09-21T07:26:21.791990+0000","last_peered":"2024-09-21T07:26:21.791990+0000","last_clean":"2024-09-21T07:26:21.791990+0000","last_became_active":"2024-09-21T07:26:14.751809+0000","last_became_peered":"2024-09-21T07:26:14.751809+0000","last_unstale":"2024-09-21T07:26:21.791990+0000","last_undegraded":"2024-09-21T07:26:21.791990+0000","last_fullsized":"2024-09-21T07:26:21.791990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:30.055064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218154+0000","last_change":"2024-09-21T07:26:19.784193+0000","last_active":"2024-09-21T07:26:23.218154+0000","last_peered":"2024-09-21T07:26:23.218154+0000","last_clean":"2024-09-21T07:26:23.218154+0000","last_became_active":"2024-09-21T07:26:19.783666+0000","last_became_peered":"2024-09-21T07:26:19.783666+0000","last_unstale":"2024-09-21T07:26:23.218154+0000","last_undegraded":"2024-09-21T07:26:23.218154+0000","last_fullsized":"2024-09-21T07:26:23.218154+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218051+0000","last_change":"2024-09-21T07:26:15.761293+0000","last_active":"2024-09-21T07:26:23.218051+0000","last_peered":"2024-09-21T07:26:23.218051+0000","last_clean":"2024-09-21T07:26:23.218051+0000","last_became_active":"2024-09-21T07:26:15.760959+0000","last_became_peered":"2024-09-21T07:26:15.760959+0000","last_unstale":"2024-09-21T07:26:23.218051+0000","last_undegraded":"2024-09-21T07:26:23.218051+0000","last_fullsized":"2024-09-21T07:26:23.218051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:12.766792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218529+0000","last_change":"2024-09-21T07:26:14.737676+0000","last_active":"2024-09-21T07:26:23.218529+0000","last_peered":"2024-09-21T07:26:23.218529+0000","last_clean":"2024-09-21T07:26:23.218529+0000","last_became_active":"2024-09-21T07:26:14.737367+0000","last_became_peered":"2024-09-21T07:26:14.737367+0000","last_unstale":"2024-09-21T07:26:23.218529+0000","last_undegraded":"2024-09-21T07:26:23.218529+0000","last_fullsized":"2024-09-21T07:26:23.218529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217727+0000","last_change":"2024-09-21T07:26:19.782689+0000","last_active":"2024-09-21T07:26:23.217727+0000","last_peered":"2024-09-21T07:26:23.217727+0000","last_clean":"2024-09-21T07:26:23.217727+0000","last_became_active":"2024-09-21T07:26:19.782215+0000","last_became_peered":"2024-09-21T07:26:19.782215+0000","last_unstale":"2024-09-21T07:26:23.217727+0000","last_undegraded":"2024-09-21T07:26:23.217727+0000","last_fullsized":"2024-09-21T07:26:23.217727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218798+0000","last_change":"2024-09-21T07:26:15.758262+0000","last_active":"2024-09-21T07:26:23.218798+0000","last_peered":"2024-09-21T07:26:23.218798+0000","last_clean":"2024-09-21T07:26:23.218798+0000","last_became_active":"2024-09-21T07:26:15.757944+0000","last_became_peered":"2024-09-21T07:26:15.757944+0000","last_unstale":"2024-09-21T07:26:23.218798+0000","last_undegraded":"2024-09-21T07:26:23.218798+0000","last_fullsized":"2024-09-21T07:26:23.218798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:10.582121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216341+0000","last_change":"2024-09-21T07:26:14.745239+0000","last_active":"2024-09-21T07:26:23.216341+0000","last_peered":"2024-09-21T07:26:23.216341+0000","last_clean":"2024-09-21T07:26:23.216341+0000","last_became_active":"2024-09-21T07:26:14.744901+0000","last_became_peered":"2024-09-21T07:26:14.744901+0000","last_unstale":"2024-09-21T07:26:23.216341+0000","last_undegraded":"2024-09-21T07:26:23.216341+0000","last_fullsized":"2024-09-21T07:26:23.216341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:07.561158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218435+0000","last_change":"2024-09-21T07:26:20.205294+0000","last_active":"2024-09-21T07:26:23.218435+0000","last_peered":"2024-09-21T07:26:23.218435+0000","last_clean":"2024-09-21T07:26:23.218435+0000","last_became_active":"2024-09-21T07:26:20.204855+0000","last_became_peered":"2024-09-21T07:26:20.204855+0000","last_unstale":"2024-09-21T07:26:23.218435+0000","last_undegraded":"2024-09-21T07:26:23.218435+0000","last_fullsized":"2024-09-21T07:26:23.218435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217428+0000","last_change":"2024-09-21T07:26:15.762544+0000","last_active":"2024-09-21T07:26:23.217428+0000","last_peered":"2024-09-21T07:26:23.217428+0000","last_clean":"2024-09-21T07:26:23.217428+0000","last_became_active":"2024-09-21T07:26:15.762193+0000","last_became_peered":"2024-09-21T07:26:15.762193+0000","last_unstale":"2024-09-21T07:26:23.217428+0000","last_undegraded":"2024-09-21T07:26:23.217428+0000","last_fullsized":"2024-09-21T07:26:23.217428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:42.681607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790143+0000","last_change":"2024-09-21T07:26:14.748084+0000","last_active":"2024-09-21T07:26:21.790143+0000","last_peered":"2024-09-21T07:26:21.790143+0000","last_clean":"2024-09-21T07:26:21.790143+0000","last_became_active":"2024-09-21T07:26:14.747288+0000","last_became_peered":"2024-09-21T07:26:14.747288+0000","last_unstale":"2024-09-21T07:26:21.790143+0000","last_undegraded":"2024-09-21T07:26:21.790143+0000","last_fullsized":"2024-09-21T07:26:21.790143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:39.950583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:19.788640+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:19.788072+0000","last_became_peered":"2024-09-21T07:26:19.788072+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756286+0000","last_change":"2024-09-21T07:26:15.757686+0000","last_active":"2024-09-21T07:26:22.756286+0000","last_peered":"2024-09-21T07:26:22.756286+0000","last_clean":"2024-09-21T07:26:22.756286+0000","last_became_active":"2024-09-21T07:26:15.757361+0000","last_became_peered":"2024-09-21T07:26:15.757361+0000","last_unstale":"2024-09-21T07:26:22.756286+0000","last_undegraded":"2024-09-21T07:26:22.756286+0000","last_fullsized":"2024-09-21T07:26:22.756286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:49.495817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791712+0000","last_change":"2024-09-21T07:26:14.746130+0000","last_active":"2024-09-21T07:26:21.791712+0000","last_peered":"2024-09-21T07:26:21.791712+0000","last_clean":"2024-09-21T07:26:21.791712+0000","last_became_active":"2024-09-21T07:26:14.745763+0000","last_became_peered":"2024-09-21T07:26:14.745763+0000","last_unstale":"2024-09-21T07:26:21.791712+0000","last_undegraded":"2024-09-21T07:26:21.791712+0000","last_fullsized":"2024-09-21T07:26:21.791712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217052+0000","last_change":"2024-09-21T07:26:20.203703+0000","last_active":"2024-09-21T07:26:23.217052+0000","last_peered":"2024-09-21T07:26:23.217052+0000","last_clean":"2024-09-21T07:26:23.217052+0000","last_became_active":"2024-09-21T07:26:20.202561+0000","last_became_peered":"2024-09-21T07:26:20.202561+0000","last_unstale":"2024-09-21T07:26:23.217052+0000","last_undegraded":"2024-09-21T07:26:23.217052+0000","last_fullsized":"2024-09-21T07:26:23.217052+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:27.041439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218313+0000","last_change":"2024-09-21T07:26:15.768378+0000","last_active":"2024-09-21T07:26:23.218313+0000","last_peered":"2024-09-21T07:26:23.218313+0000","last_clean":"2024-09-21T07:26:23.218313+0000","last_became_active":"2024-09-21T07:26:15.768068+0000","last_became_peered":"2024-09-21T07:26:15.768068+0000","last_unstale":"2024-09-21T07:26:23.218313+0000","last_undegraded":"2024-09-21T07:26:23.218313+0000","last_fullsized":"2024-09-21T07:26:23.218313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:12.636547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216461+0000","last_change":"2024-09-21T07:26:14.741727+0000","last_active":"2024-09-21T07:26:23.216461+0000","last_peered":"2024-09-21T07:26:23.216461+0000","last_clean":"2024-09-21T07:26:23.216461+0000","last_became_active":"2024-09-21T07:26:14.741372+0000","last_became_peered":"2024-09-21T07:26:14.741372+0000","last_unstale":"2024-09-21T07:26:23.216461+0000","last_undegraded":"2024-09-21T07:26:23.216461+0000","last_fullsized":"2024-09-21T07:26:23.216461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:48.123776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218506+0000","last_change":"2024-09-21T07:26:20.203003+0000","last_active":"2024-09-21T07:26:23.218506+0000","last_peered":"2024-09-21T07:26:23.218506+0000","last_clean":"2024-09-21T07:26:23.218506+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:23.218506+0000","last_undegraded":"2024-09-21T07:26:23.218506+0000","last_fullsized":"2024-09-21T07:26:23.218506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:12.749885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217922+0000","last_change":"2024-09-21T07:26:15.750763+0000","last_active":"2024-09-21T07:26:23.217922+0000","last_peered":"2024-09-21T07:26:23.217922+0000","last_clean":"2024-09-21T07:26:23.217922+0000","last_became_active":"2024-09-21T07:26:15.750444+0000","last_became_peered":"2024-09-21T07:26:15.750444+0000","last_unstale":"2024-09-21T07:26:23.217922+0000","last_undegraded":"2024-09-21T07:26:23.217922+0000","last_fullsized":"2024-09-21T07:26:23.217922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792041+0000","last_change":"2024-09-21T07:26:14.745457+0000","last_active":"2024-09-21T07:26:21.792041+0000","last_peered":"2024-09-21T07:26:21.792041+0000","last_clean":"2024-09-21T07:26:21.792041+0000","last_became_active":"2024-09-21T07:26:14.745075+0000","last_became_peered":"2024-09-21T07:26:14.745075+0000","last_unstale":"2024-09-21T07:26:21.792041+0000","last_undegraded":"2024-09-21T07:26:21.792041+0000","last_fullsized":"2024-09-21T07:26:21.792041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:55:49.174372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756061+0000","last_change":"2024-09-21T07:26:20.202406+0000","last_active":"2024-09-21T07:26:22.756061+0000","last_peered":"2024-09-21T07:26:22.756061+0000","last_clean":"2024-09-21T07:26:22.756061+0000","last_became_active":"2024-09-21T07:26:20.201970+0000","last_became_peered":"2024-09-21T07:26:20.201970+0000","last_unstale":"2024-09-21T07:26:22.756061+0000","last_undegraded":"2024-09-21T07:26:22.756061+0000","last_fullsized":"2024-09-21T07:26:22.756061+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901194+0000","last_change":"2024-09-21T07:26:15.769492+0000","last_active":"2024-09-21T07:26:22.901194+0000","last_peered":"2024-09-21T07:26:22.901194+0000","last_clean":"2024-09-21T07:26:22.901194+0000","last_became_active":"2024-09-21T07:26:15.768661+0000","last_became_peered":"2024-09-21T07:26:15.768661+0000","last_unstale":"2024-09-21T07:26:22.901194+0000","last_undegraded":"2024-09-21T07:26:22.901194+0000","last_fullsized":"2024-09-21T07:26:22.901194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:09:20.108036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218024+0000","last_change":"2024-09-21T07:26:14.737203+0000","last_active":"2024-09-21T07:26:23.218024+0000","last_peered":"2024-09-21T07:26:23.218024+0000","last_clean":"2024-09-21T07:26:23.218024+0000","last_became_active":"2024-09-21T07:26:14.736387+0000","last_became_peered":"2024-09-21T07:26:14.736387+0000","last_unstale":"2024-09-21T07:26:23.218024+0000","last_undegraded":"2024-09-21T07:26:23.218024+0000","last_fullsized":"2024-09-21T07:26:23.218024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217332+0000","last_change":"2024-09-21T07:26:19.793567+0000","last_active":"2024-09-21T07:26:23.217332+0000","last_peered":"2024-09-21T07:26:23.217332+0000","last_clean":"2024-09-21T07:26:23.217332+0000","last_became_active":"2024-09-21T07:26:19.793063+0000","last_became_peered":"2024-09-21T07:26:19.793063+0000","last_unstale":"2024-09-21T07:26:23.217332+0000","last_undegraded":"2024-09-21T07:26:23.217332+0000","last_fullsized":"2024-09-21T07:26:23.217332+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218020+0000","last_change":"2024-09-21T07:26:15.764293+0000","last_active":"2024-09-21T07:26:23.218020+0000","last_peered":"2024-09-21T07:26:23.218020+0000","last_clean":"2024-09-21T07:26:23.218020+0000","last_became_active":"2024-09-21T07:26:15.763942+0000","last_became_peered":"2024-09-21T07:26:15.763942+0000","last_unstale":"2024-09-21T07:26:23.218020+0000","last_undegraded":"2024-09-21T07:26:23.218020+0000","last_fullsized":"2024-09-21T07:26:23.218020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:40.957814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900887+0000","last_change":"2024-09-21T07:26:14.753757+0000","last_active":"2024-09-21T07:26:22.900887+0000","last_peered":"2024-09-21T07:26:22.900887+0000","last_clean":"2024-09-21T07:26:22.900887+0000","last_became_active":"2024-09-21T07:26:14.753415+0000","last_became_peered":"2024-09-21T07:26:14.753415+0000","last_unstale":"2024-09-21T07:26:22.900887+0000","last_undegraded":"2024-09-21T07:26:22.900887+0000","last_fullsized":"2024-09-21T07:26:22.900887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:48.397593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217762+0000","last_change":"2024-09-21T07:26:19.786373+0000","last_active":"2024-09-21T07:26:23.217762+0000","last_peered":"2024-09-21T07:26:23.217762+0000","last_clean":"2024-09-21T07:26:23.217762+0000","last_became_active":"2024-09-21T07:26:19.785794+0000","last_became_peered":"2024-09-21T07:26:19.785794+0000","last_unstale":"2024-09-21T07:26:23.217762+0000","last_undegraded":"2024-09-21T07:26:23.217762+0000","last_fullsized":"2024-09-21T07:26:23.217762+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:06.296352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216872+0000","last_change":"2024-09-21T07:26:15.752034+0000","last_active":"2024-09-21T07:26:23.216872+0000","last_peered":"2024-09-21T07:26:23.216872+0000","last_clean":"2024-09-21T07:26:23.216872+0000","last_became_active":"2024-09-21T07:26:15.751733+0000","last_became_peered":"2024-09-21T07:26:15.751733+0000","last_unstale":"2024-09-21T07:26:23.216872+0000","last_undegraded":"2024-09-21T07:26:23.216872+0000","last_fullsized":"2024-09-21T07:26:23.216872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789086+0000","last_change":"2024-09-21T07:26:14.741318+0000","last_active":"2024-09-21T07:26:21.789086+0000","last_peered":"2024-09-21T07:26:21.789086+0000","last_clean":"2024-09-21T07:26:21.789086+0000","last_became_active":"2024-09-21T07:26:14.740828+0000","last_became_peered":"2024-09-21T07:26:14.740828+0000","last_unstale":"2024-09-21T07:26:21.789086+0000","last_undegraded":"2024-09-21T07:26:21.789086+0000","last_fullsized":"2024-09-21T07:26:21.789086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:09.614543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218456+0000","last_change":"2024-09-21T07:26:19.793288+0000","last_active":"2024-09-21T07:26:23.218456+0000","last_peered":"2024-09-21T07:26:23.218456+0000","last_clean":"2024-09-21T07:26:23.218456+0000","last_became_active":"2024-09-21T07:26:19.792781+0000","last_became_peered":"2024-09-21T07:26:19.792781+0000","last_unstale":"2024-09-21T07:26:23.218456+0000","last_undegraded":"2024-09-21T07:26:23.218456+0000","last_fullsized":"2024-09-21T07:26:23.218456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755586+0000","last_change":"2024-09-21T07:26:15.763289+0000","last_active":"2024-09-21T07:26:22.755586+0000","last_peered":"2024-09-21T07:26:22.755586+0000","last_clean":"2024-09-21T07:26:22.755586+0000","last_became_active":"2024-09-21T07:26:15.762914+0000","last_became_peered":"2024-09-21T07:26:15.762914+0000","last_unstale":"2024-09-21T07:26:22.755586+0000","last_undegraded":"2024-09-21T07:26:22.755586+0000","last_fullsized":"2024-09-21T07:26:22.755586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:01.890170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790269+0000","last_change":"2024-09-21T07:26:14.746676+0000","last_active":"2024-09-21T07:26:21.790269+0000","last_peered":"2024-09-21T07:26:21.790269+0000","last_clean":"2024-09-21T07:26:21.790269+0000","last_became_active":"2024-09-21T07:26:14.745094+0000","last_became_peered":"2024-09-21T07:26:14.745094+0000","last_unstale":"2024-09-21T07:26:21.790269+0000","last_undegraded":"2024-09-21T07:26:21.790269+0000","last_fullsized":"2024-09-21T07:26:21.790269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:48.031389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756099+0000","last_change":"2024-09-21T07:26:20.202272+0000","last_active":"2024-09-21T07:26:22.756099+0000","last_peered":"2024-09-21T07:26:22.756099+0000","last_clean":"2024-09-21T07:26:22.756099+0000","last_became_active":"2024-09-21T07:26:20.201710+0000","last_became_peered":"2024-09-21T07:26:20.201710+0000","last_unstale":"2024-09-21T07:26:22.756099+0000","last_undegraded":"2024-09-21T07:26:22.756099+0000","last_fullsized":"2024-09-21T07:26:22.756099+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:26.267566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217063+0000","last_change":"2024-09-21T07:26:15.778494+0000","last_active":"2024-09-21T07:26:23.217063+0000","last_peered":"2024-09-21T07:26:23.217063+0000","last_clean":"2024-09-21T07:26:23.217063+0000","last_became_active":"2024-09-21T07:26:15.778117+0000","last_became_peered":"2024-09-21T07:26:15.778117+0000","last_unstale":"2024-09-21T07:26:23.217063+0000","last_undegraded":"2024-09-21T07:26:23.217063+0000","last_fullsized":"2024-09-21T07:26:23.217063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:52.614175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900190+0000","last_change":"2024-09-21T07:26:14.755827+0000","last_active":"2024-09-21T07:26:22.900190+0000","last_peered":"2024-09-21T07:26:22.900190+0000","last_clean":"2024-09-21T07:26:22.900190+0000","last_became_active":"2024-09-21T07:26:14.755319+0000","last_became_peered":"2024-09-21T07:26:14.755319+0000","last_unstale":"2024-09-21T07:26:22.900190+0000","last_undegraded":"2024-09-21T07:26:22.900190+0000","last_fullsized":"2024-09-21T07:26:22.900190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:31.306111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790381+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:26:21.790381+0000","last_peered":"2024-09-21T07:26:21.790381+0000","last_clean":"2024-09-21T07:26:21.790381+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:26:21.790381+0000","last_undegraded":"2024-09-21T07:26:21.790381+0000","last_fullsized":"2024-09-21T07:26:21.790381+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790882+0000","last_change":"2024-09-21T07:26:20.205543+0000","last_active":"2024-09-21T07:26:21.790882+0000","last_peered":"2024-09-21T07:26:21.790882+0000","last_clean":"2024-09-21T07:26:21.790882+0000","last_became_active":"2024-09-21T07:26:20.204540+0000","last_became_peered":"2024-09-21T07:26:20.204540+0000","last_unstale":"2024-09-21T07:26:21.790882+0000","last_undegraded":"2024-09-21T07:26:21.790882+0000","last_fullsized":"2024-09-21T07:26:21.790882+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:35.987966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791526+0000","last_change":"2024-09-21T07:26:15.776215+0000","last_active":"2024-09-21T07:26:21.791526+0000","last_peered":"2024-09-21T07:26:21.791526+0000","last_clean":"2024-09-21T07:26:21.791526+0000","last_became_active":"2024-09-21T07:26:15.775884+0000","last_became_peered":"2024-09-21T07:26:15.775884+0000","last_unstale":"2024-09-21T07:26:21.791526+0000","last_undegraded":"2024-09-21T07:26:21.791526+0000","last_fullsized":"2024-09-21T07:26:21.791526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:01.235148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789487+0000","last_change":"2024-09-21T07:26:14.756009+0000","last_active":"2024-09-21T07:26:21.789487+0000","last_peered":"2024-09-21T07:26:21.789487+0000","last_clean":"2024-09-21T07:26:21.789487+0000","last_became_active":"2024-09-21T07:26:14.755653+0000","last_became_peered":"2024-09-21T07:26:14.755653+0000","last_unstale":"2024-09-21T07:26:21.789487+0000","last_undegraded":"2024-09-21T07:26:21.789487+0000","last_fullsized":"2024-09-21T07:26:21.789487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:22.235227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790353+0000","last_change":"2024-09-21T07:26:20.202159+0000","last_active":"2024-09-21T07:26:21.790353+0000","last_peered":"2024-09-21T07:26:21.790353+0000","last_clean":"2024-09-21T07:26:21.790353+0000","last_became_active":"2024-09-21T07:26:20.201680+0000","last_became_peered":"2024-09-21T07:26:20.201680+0000","last_unstale":"2024-09-21T07:26:21.790353+0000","last_undegraded":"2024-09-21T07:26:21.790353+0000","last_fullsized":"2024-09-21T07:26:21.790353+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:59:56.792767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901045+0000","last_change":"2024-09-21T07:26:15.771710+0000","last_active":"2024-09-21T07:26:22.901045+0000","last_peered":"2024-09-21T07:26:22.901045+0000","last_clean":"2024-09-21T07:26:22.901045+0000","last_became_active":"2024-09-21T07:26:15.770649+0000","last_became_peered":"2024-09-21T07:26:15.770649+0000","last_unstale":"2024-09-21T07:26:22.901045+0000","last_undegraded":"2024-09-21T07:26:22.901045+0000","last_fullsized":"2024-09-21T07:26:22.901045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:45.904168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218377+0000","last_change":"2024-09-21T07:26:14.740185+0000","last_active":"2024-09-21T07:26:23.218377+0000","last_peered":"2024-09-21T07:26:23.218377+0000","last_clean":"2024-09-21T07:26:23.218377+0000","last_became_active":"2024-09-21T07:26:14.738935+0000","last_became_peered":"2024-09-21T07:26:14.738935+0000","last_unstale":"2024-09-21T07:26:23.218377+0000","last_undegraded":"2024-09-21T07:26:23.218377+0000","last_fullsized":"2024-09-21T07:26:23.218377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:44.566128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756214+0000","last_change":"2024-09-21T07:26:19.789320+0000","last_active":"2024-09-21T07:26:22.756214+0000","last_peered":"2024-09-21T07:26:22.756214+0000","last_clean":"2024-09-21T07:26:22.756214+0000","last_became_active":"2024-09-21T07:26:19.788692+0000","last_became_peered":"2024-09-21T07:26:19.788692+0000","last_unstale":"2024-09-21T07:26:22.756214+0000","last_undegraded":"2024-09-21T07:26:22.756214+0000","last_fullsized":"2024-09-21T07:26:22.756214+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:18:05.320183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792007+0000","last_change":"2024-09-21T07:26:15.775282+0000","last_active":"2024-09-21T07:26:21.792007+0000","last_peered":"2024-09-21T07:26:21.792007+0000","last_clean":"2024-09-21T07:26:21.792007+0000","last_became_active":"2024-09-21T07:26:15.775046+0000","last_became_peered":"2024-09-21T07:26:15.775046+0000","last_unstale":"2024-09-21T07:26:21.792007+0000","last_undegraded":"2024-09-21T07:26:21.792007+0000","last_fullsized":"2024-09-21T07:26:21.792007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:51.588092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789401+0000","last_change":"2024-09-21T07:26:14.740365+0000","last_active":"2024-09-21T07:26:21.789401+0000","last_peered":"2024-09-21T07:26:21.789401+0000","last_clean":"2024-09-21T07:26:21.789401+0000","last_became_active":"2024-09-21T07:26:14.739987+0000","last_became_peered":"2024-09-21T07:26:14.739987+0000","last_unstale":"2024-09-21T07:26:21.789401+0000","last_undegraded":"2024-09-21T07:26:21.789401+0000","last_fullsized":"2024-09-21T07:26:21.789401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:41.674376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900421+0000","last_change":"2024-09-21T07:26:20.205672+0000","last_active":"2024-09-21T07:26:22.900421+0000","last_peered":"2024-09-21T07:26:22.900421+0000","last_clean":"2024-09-21T07:26:22.900421+0000","last_became_active":"2024-09-21T07:26:20.205251+0000","last_became_peered":"2024-09-21T07:26:20.205251+0000","last_unstale":"2024-09-21T07:26:22.900421+0000","last_undegraded":"2024-09-21T07:26:22.900421+0000","last_fullsized":"2024-09-21T07:26:22.900421+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:18.155040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790765+0000","last_change":"2024-09-21T07:26:15.762918+0000","last_active":"2024-09-21T07:26:21.790765+0000","last_peered":"2024-09-21T07:26:21.790765+0000","last_clean":"2024-09-21T07:26:21.790765+0000","last_became_active":"2024-09-21T07:26:15.762580+0000","last_became_peered":"2024-09-21T07:26:15.762580+0000","last_unstale":"2024-09-21T07:26:21.790765+0000","last_undegraded":"2024-09-21T07:26:21.790765+0000","last_fullsized":"2024-09-21T07:26:21.790765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:47.691687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218021+0000","last_change":"2024-09-21T07:26:14.739242+0000","last_active":"2024-09-21T07:26:23.218021+0000","last_peered":"2024-09-21T07:26:23.218021+0000","last_clean":"2024-09-21T07:26:23.218021+0000","last_became_active":"2024-09-21T07:26:14.737910+0000","last_became_peered":"2024-09-21T07:26:14.737910+0000","last_unstale":"2024-09-21T07:26:23.218021+0000","last_undegraded":"2024-09-21T07:26:23.218021+0000","last_fullsized":"2024-09-21T07:26:23.218021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:30.239449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217037+0000","last_change":"2024-09-21T07:26:19.782735+0000","last_active":"2024-09-21T07:26:23.217037+0000","last_peered":"2024-09-21T07:26:23.217037+0000","last_clean":"2024-09-21T07:26:23.217037+0000","last_became_active":"2024-09-21T07:26:19.782062+0000","last_became_peered":"2024-09-21T07:26:19.782062+0000","last_unstale":"2024-09-21T07:26:23.217037+0000","last_undegraded":"2024-09-21T07:26:23.217037+0000","last_fullsized":"2024-09-21T07:26:23.217037+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:05.271854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790993+0000","last_change":"2024-09-21T07:26:15.772188+0000","last_active":"2024-09-21T07:26:21.790993+0000","last_peered":"2024-09-21T07:26:21.790993+0000","last_clean":"2024-09-21T07:26:21.790993+0000","last_became_active":"2024-09-21T07:26:15.771854+0000","last_became_peered":"2024-09-21T07:26:15.771854+0000","last_unstale":"2024-09-21T07:26:21.790993+0000","last_undegraded":"2024-09-21T07:26:21.790993+0000","last_fullsized":"2024-09-21T07:26:21.790993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:52.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.788951+0000","last_change":"2024-09-21T07:26:14.739783+0000","last_active":"2024-09-21T07:26:21.788951+0000","last_peered":"2024-09-21T07:26:21.788951+0000","last_clean":"2024-09-21T07:26:21.788951+0000","last_became_active":"2024-09-21T07:26:14.739433+0000","last_became_peered":"2024-09-21T07:26:14.739433+0000","last_unstale":"2024-09-21T07:26:21.788951+0000","last_undegraded":"2024-09-21T07:26:21.788951+0000","last_fullsized":"2024-09-21T07:26:21.788951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:08.827173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756045+0000","last_change":"2024-09-21T07:26:19.789353+0000","last_active":"2024-09-21T07:26:22.756045+0000","last_peered":"2024-09-21T07:26:22.756045+0000","last_clean":"2024-09-21T07:26:22.756045+0000","last_became_active":"2024-09-21T07:26:19.788890+0000","last_became_peered":"2024-09-21T07:26:19.788890+0000","last_unstale":"2024-09-21T07:26:22.756045+0000","last_undegraded":"2024-09-21T07:26:22.756045+0000","last_fullsized":"2024-09-21T07:26:22.756045+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790254+0000","last_change":"2024-09-21T07:26:15.761846+0000","last_active":"2024-09-21T07:26:21.790254+0000","last_peered":"2024-09-21T07:26:21.790254+0000","last_clean":"2024-09-21T07:26:21.790254+0000","last_became_active":"2024-09-21T07:26:15.761125+0000","last_became_peered":"2024-09-21T07:26:15.761125+0000","last_unstale":"2024-09-21T07:26:21.790254+0000","last_undegraded":"2024-09-21T07:26:21.790254+0000","last_fullsized":"2024-09-21T07:26:21.790254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791303+0000","last_change":"2024-09-21T07:26:14.751611+0000","last_active":"2024-09-21T07:26:21.791303+0000","last_peered":"2024-09-21T07:26:21.791303+0000","last_clean":"2024-09-21T07:26:21.791303+0000","last_became_active":"2024-09-21T07:26:14.750246+0000","last_became_peered":"2024-09-21T07:26:14.750246+0000","last_unstale":"2024-09-21T07:26:21.791303+0000","last_undegraded":"2024-09-21T07:26:21.791303+0000","last_fullsized":"2024-09-21T07:26:21.791303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:45.761871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218074+0000","last_change":"2024-09-21T07:26:20.202567+0000","last_active":"2024-09-21T07:26:23.218074+0000","last_peered":"2024-09-21T07:26:23.218074+0000","last_clean":"2024-09-21T07:26:23.218074+0000","last_became_active":"2024-09-21T07:26:20.201637+0000","last_became_peered":"2024-09-21T07:26:20.201637+0000","last_unstale":"2024-09-21T07:26:23.218074+0000","last_undegraded":"2024-09-21T07:26:23.218074+0000","last_fullsized":"2024-09-21T07:26:23.218074+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:17.143053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791707+0000","last_change":"2024-09-21T07:26:15.775390+0000","last_active":"2024-09-21T07:26:21.791707+0000","last_peered":"2024-09-21T07:26:21.791707+0000","last_clean":"2024-09-21T07:26:21.791707+0000","last_became_active":"2024-09-21T07:26:15.774972+0000","last_became_peered":"2024-09-21T07:26:15.774972+0000","last_unstale":"2024-09-21T07:26:21.791707+0000","last_undegraded":"2024-09-21T07:26:21.791707+0000","last_fullsized":"2024-09-21T07:26:21.791707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.551786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900819+0000","last_change":"2024-09-21T07:26:14.752809+0000","last_active":"2024-09-21T07:26:22.900819+0000","last_peered":"2024-09-21T07:26:22.900819+0000","last_clean":"2024-09-21T07:26:22.900819+0000","last_became_active":"2024-09-21T07:26:14.752442+0000","last_became_peered":"2024-09-21T07:26:14.752442+0000","last_unstale":"2024-09-21T07:26:22.900819+0000","last_undegraded":"2024-09-21T07:26:22.900819+0000","last_fullsized":"2024-09-21T07:26:22.900819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:52.370404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218055+0000","last_change":"2024-09-21T07:26:20.202097+0000","last_active":"2024-09-21T07:26:23.218055+0000","last_peered":"2024-09-21T07:26:23.218055+0000","last_clean":"2024-09-21T07:26:23.218055+0000","last_became_active":"2024-09-21T07:26:20.201641+0000","last_became_peered":"2024-09-21T07:26:20.201641+0000","last_unstale":"2024-09-21T07:26:23.218055+0000","last_undegraded":"2024-09-21T07:26:23.218055+0000","last_fullsized":"2024-09-21T07:26:23.218055+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:52.838818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755771+0000","last_change":"2024-09-21T07:26:15.763851+0000","last_active":"2024-09-21T07:26:22.755771+0000","last_peered":"2024-09-21T07:26:22.755771+0000","last_clean":"2024-09-21T07:26:22.755771+0000","last_became_active":"2024-09-21T07:26:15.763088+0000","last_became_peered":"2024-09-21T07:26:15.763088+0000","last_unstale":"2024-09-21T07:26:22.755771+0000","last_undegraded":"2024-09-21T07:26:22.755771+0000","last_fullsized":"2024-09-21T07:26:22.755771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:00.411908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756372+0000","last_change":"2024-09-21T07:26:14.749147+0000","last_active":"2024-09-21T07:26:22.756372+0000","last_peered":"2024-09-21T07:26:22.756372+0000","last_clean":"2024-09-21T07:26:22.756372+0000","last_became_active":"2024-09-21T07:26:14.748858+0000","last_became_peered":"2024-09-21T07:26:14.748858+0000","last_unstale":"2024-09-21T07:26:22.756372+0000","last_undegraded":"2024-09-21T07:26:22.756372+0000","last_fullsized":"2024-09-21T07:26:22.756372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:19.994800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901148+0000","last_change":"2024-09-21T07:26:20.202180+0000","last_active":"2024-09-21T07:26:22.901148+0000","last_peered":"2024-09-21T07:26:22.901148+0000","last_clean":"2024-09-21T07:26:22.901148+0000","last_became_active":"2024-09-21T07:26:20.201665+0000","last_became_peered":"2024-09-21T07:26:20.201665+0000","last_unstale":"2024-09-21T07:26:22.901148+0000","last_undegraded":"2024-09-21T07:26:22.901148+0000","last_fullsized":"2024-09-21T07:26:22.901148+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:49.726125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.763407+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.763087+0000","last_became_peered":"2024-09-21T07:26:15.763087+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:49.061741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:26.613007+0000","last_change":"2024-09-21T07:26:14.746399+0000","last_active":"2024-09-21T07:26:26.613007+0000","last_peered":"2024-09-21T07:26:26.613007+0000","last_clean":"2024-09-21T07:26:26.613007+0000","last_became_active":"2024-09-21T07:26:14.746029+0000","last_became_peered":"2024-09-21T07:26:14.746029+0000","last_unstale":"2024-09-21T07:26:26.613007+0000","last_undegraded":"2024-09-21T07:26:26.613007+0000","last_fullsized":"2024-09-21T07:26:26.613007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:47.548805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217015+0000","last_change":"2024-09-21T07:26:19.793971+0000","last_active":"2024-09-21T07:26:23.217015+0000","last_peered":"2024-09-21T07:26:23.217015+0000","last_clean":"2024-09-21T07:26:23.217015+0000","last_became_active":"2024-09-21T07:26:19.793342+0000","last_became_peered":"2024-09-21T07:26:19.793342+0000","last_unstale":"2024-09-21T07:26:23.217015+0000","last_undegraded":"2024-09-21T07:26:23.217015+0000","last_fullsized":"2024-09-21T07:26:23.217015+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217997+0000","last_change":"2024-09-21T07:26:15.763698+0000","last_active":"2024-09-21T07:26:23.217997+0000","last_peered":"2024-09-21T07:26:23.217997+0000","last_clean":"2024-09-21T07:26:23.217997+0000","last_became_active":"2024-09-21T07:26:15.763010+0000","last_became_peered":"2024-09-21T07:26:15.763010+0000","last_unstale":"2024-09-21T07:26:23.217997+0000","last_undegraded":"2024-09-21T07:26:23.217997+0000","last_fullsized":"2024-09-21T07:26:23.217997+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:49.038621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218340+0000","last_change":"2024-09-21T07:26:14.756162+0000","last_active":"2024-09-21T07:26:23.218340+0000","last_peered":"2024-09-21T07:26:23.218340+0000","last_clean":"2024-09-21T07:26:23.218340+0000","last_became_active":"2024-09-21T07:26:14.755821+0000","last_became_peered":"2024-09-21T07:26:14.755821+0000","last_unstale":"2024-09-21T07:26:23.218340+0000","last_undegraded":"2024-09-21T07:26:23.218340+0000","last_fullsized":"2024-09-21T07:26:23.218340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217766+0000","last_change":"2024-09-21T07:26:19.782715+0000","last_active":"2024-09-21T07:26:23.217766+0000","last_peered":"2024-09-21T07:26:23.217766+0000","last_clean":"2024-09-21T07:26:23.217766+0000","last_became_active":"2024-09-21T07:26:19.782285+0000","last_became_peered":"2024-09-21T07:26:19.782285+0000","last_unstale":"2024-09-21T07:26:23.217766+0000","last_undegraded":"2024-09-21T07:26:23.217766+0000","last_fullsized":"2024-09-21T07:26:23.217766+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:21:57.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756175+0000","last_change":"2024-09-21T07:26:15.759805+0000","last_active":"2024-09-21T07:26:22.756175+0000","last_peered":"2024-09-21T07:26:22.756175+0000","last_clean":"2024-09-21T07:26:22.756175+0000","last_became_active":"2024-09-21T07:26:15.759487+0000","last_became_peered":"2024-09-21T07:26:15.759487+0000","last_unstale":"2024-09-21T07:26:22.756175+0000","last_undegraded":"2024-09-21T07:26:22.756175+0000","last_fullsized":"2024-09-21T07:26:22.756175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900206+0000","last_change":"2024-09-21T07:26:14.753655+0000","last_active":"2024-09-21T07:26:22.900206+0000","last_peered":"2024-09-21T07:26:22.900206+0000","last_clean":"2024-09-21T07:26:22.900206+0000","last_became_active":"2024-09-21T07:26:14.753266+0000","last_became_peered":"2024-09-21T07:26:14.753266+0000","last_unstale":"2024-09-21T07:26:22.900206+0000","last_undegraded":"2024-09-21T07:26:22.900206+0000","last_fullsized":"2024-09-21T07:26:22.900206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:14.462696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218754+0000","last_change":"2024-09-21T07:26:19.786292+0000","last_active":"2024-09-21T07:26:23.218754+0000","last_peered":"2024-09-21T07:26:23.218754+0000","last_clean":"2024-09-21T07:26:23.218754+0000","last_became_active":"2024-09-21T07:26:19.783800+0000","last_became_peered":"2024-09-21T07:26:19.783800+0000","last_unstale":"2024-09-21T07:26:23.218754+0000","last_undegraded":"2024-09-21T07:26:23.218754+0000","last_fullsized":"2024-09-21T07:26:23.218754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:13:52.992488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901112+0000","last_change":"2024-09-21T07:26:15.766957+0000","last_active":"2024-09-21T07:26:22.901112+0000","last_peered":"2024-09-21T07:26:22.901112+0000","last_clean":"2024-09-21T07:26:22.901112+0000","last_became_active":"2024-09-21T07:26:15.766640+0000","last_became_peered":"2024-09-21T07:26:15.766640+0000","last_unstale":"2024-09-21T07:26:22.901112+0000","last_undegraded":"2024-09-21T07:26:22.901112+0000","last_fullsized":"2024-09-21T07:26:22.901112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:23.200622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791558+0000","last_change":"2024-09-21T07:26:14.756852+0000","last_active":"2024-09-21T07:26:21.791558+0000","last_peered":"2024-09-21T07:26:21.791558+0000","last_clean":"2024-09-21T07:26:21.791558+0000","last_became_active":"2024-09-21T07:26:14.756512+0000","last_became_peered":"2024-09-21T07:26:14.756512+0000","last_unstale":"2024-09-21T07:26:21.791558+0000","last_undegraded":"2024-09-21T07:26:21.791558+0000","last_fullsized":"2024-09-21T07:26:21.791558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:21.682014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217003+0000","last_change":"2024-09-21T07:26:20.205385+0000","last_active":"2024-09-21T07:26:23.217003+0000","last_peered":"2024-09-21T07:26:23.217003+0000","last_clean":"2024-09-21T07:26:23.217003+0000","last_became_active":"2024-09-21T07:26:20.205016+0000","last_became_peered":"2024-09-21T07:26:20.205016+0000","last_unstale":"2024-09-21T07:26:23.217003+0000","last_undegraded":"2024-09-21T07:26:23.217003+0000","last_fullsized":"2024-09-21T07:26:23.217003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.612156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755846+0000","last_change":"2024-09-21T07:26:20.202075+0000","last_active":"2024-09-21T07:26:22.755846+0000","last_peered":"2024-09-21T07:26:22.755846+0000","last_clean":"2024-09-21T07:26:22.755846+0000","last_became_active":"2024-09-21T07:26:20.201632+0000","last_became_peered":"2024-09-21T07:26:20.201632+0000","last_unstale":"2024-09-21T07:26:22.755846+0000","last_undegraded":"2024-09-21T07:26:22.755846+0000","last_fullsized":"2024-09-21T07:26:22.755846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.190700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216493+0000","last_change":"2024-09-21T07:26:15.762928+0000","last_active":"2024-09-21T07:26:23.216493+0000","last_peered":"2024-09-21T07:26:23.216493+0000","last_clean":"2024-09-21T07:26:23.216493+0000","last_became_active":"2024-09-21T07:26:15.762291+0000","last_became_peered":"2024-09-21T07:26:15.762291+0000","last_unstale":"2024-09-21T07:26:23.216493+0000","last_undegraded":"2024-09-21T07:26:23.216493+0000","last_fullsized":"2024-09-21T07:26:23.216493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:06.516630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900637+0000","last_change":"2024-09-21T07:26:14.755764+0000","last_active":"2024-09-21T07:26:22.900637+0000","last_peered":"2024-09-21T07:26:22.900637+0000","last_clean":"2024-09-21T07:26:22.900637+0000","last_became_active":"2024-09-21T07:26:14.755397+0000","last_became_peered":"2024-09-21T07:26:14.755397+0000","last_unstale":"2024-09-21T07:26:22.900637+0000","last_undegraded":"2024-09-21T07:26:22.900637+0000","last_fullsized":"2024-09-21T07:26:22.900637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:32.946211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791301+0000","last_change":"2024-09-21T07:26:19.789958+0000","last_active":"2024-09-21T07:26:21.791301+0000","last_peered":"2024-09-21T07:26:21.791301+0000","last_clean":"2024-09-21T07:26:21.791301+0000","last_became_active":"2024-09-21T07:26:19.789515+0000","last_became_peered":"2024-09-21T07:26:19.789515+0000","last_unstale":"2024-09-21T07:26:21.791301+0000","last_undegraded":"2024-09-21T07:26:21.791301+0000","last_fullsized":"2024-09-21T07:26:21.791301+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:25.737069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218685+0000","last_change":"2024-09-21T07:26:15.760941+0000","last_active":"2024-09-21T07:26:23.218685+0000","last_peered":"2024-09-21T07:26:23.218685+0000","last_clean":"2024-09-21T07:26:23.218685+0000","last_became_active":"2024-09-21T07:26:15.760612+0000","last_became_peered":"2024-09-21T07:26:15.760612+0000","last_unstale":"2024-09-21T07:26:23.218685+0000","last_undegraded":"2024-09-21T07:26:23.218685+0000","last_fullsized":"2024-09-21T07:26:23.218685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:23.610491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216574+0000","last_change":"2024-09-21T07:26:14.744602+0000","last_active":"2024-09-21T07:26:23.216574+0000","last_peered":"2024-09-21T07:26:23.216574+0000","last_clean":"2024-09-21T07:26:23.216574+0000","last_became_active":"2024-09-21T07:26:14.744284+0000","last_became_peered":"2024-09-21T07:26:14.744284+0000","last_unstale":"2024-09-21T07:26:23.216574+0000","last_undegraded":"2024-09-21T07:26:23.216574+0000","last_fullsized":"2024-09-21T07:26:23.216574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:39.642931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790820+0000","last_change":"2024-09-21T07:26:14.750830+0000","last_active":"2024-09-21T07:26:21.790820+0000","last_peered":"2024-09-21T07:26:21.790820+0000","last_clean":"2024-09-21T07:26:21.790820+0000","last_became_active":"2024-09-21T07:26:14.749815+0000","last_became_peered":"2024-09-21T07:26:14.749815+0000","last_unstale":"2024-09-21T07:26:21.790820+0000","last_undegraded":"2024-09-21T07:26:21.790820+0000","last_fullsized":"2024-09-21T07:26:21.790820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:45.274521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756340+0000","last_change":"2024-09-21T07:26:15.773671+0000","last_active":"2024-09-21T07:26:22.756340+0000","last_peered":"2024-09-21T07:26:22.756340+0000","last_clean":"2024-09-21T07:26:22.756340+0000","last_became_active":"2024-09-21T07:26:15.773380+0000","last_became_peered":"2024-09-21T07:26:15.773380+0000","last_unstale":"2024-09-21T07:26:22.756340+0000","last_undegraded":"2024-09-21T07:26:22.756340+0000","last_fullsized":"2024-09-21T07:26:22.756340+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789217+0000","last_change":"2024-09-21T07:26:19.785278+0000","last_active":"2024-09-21T07:26:21.789217+0000","last_peered":"2024-09-21T07:26:21.789217+0000","last_clean":"2024-09-21T07:26:21.789217+0000","last_became_active":"2024-09-21T07:26:19.784142+0000","last_became_peered":"2024-09-21T07:26:19.784142+0000","last_unstale":"2024-09-21T07:26:21.789217+0000","last_undegraded":"2024-09-21T07:26:21.789217+0000","last_fullsized":"2024-09-21T07:26:21.789217+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.144434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217614+0000","last_change":"2024-09-21T07:26:14.746012+0000","last_active":"2024-09-21T07:26:23.217614+0000","last_peered":"2024-09-21T07:26:23.217614+0000","last_clean":"2024-09-21T07:26:23.217614+0000","last_became_active":"2024-09-21T07:26:14.745668+0000","last_became_peered":"2024-09-21T07:26:14.745668+0000","last_unstale":"2024-09-21T07:26:23.217614+0000","last_undegraded":"2024-09-21T07:26:23.217614+0000","last_fullsized":"2024-09-21T07:26:23.217614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:13.714024+0000","last_clean_scrub_stamp":"2024-09-21T07:26:13.714024+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:04:03.901416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217510+0000","last_change":"2024-09-21T07:26:15.764689+0000","last_active":"2024-09-21T07:26:23.217510+0000","last_peered":"2024-09-21T07:26:23.217510+0000","last_clean":"2024-09-21T07:26:23.217510+0000","last_became_active":"2024-09-21T07:26:15.763516+0000","last_became_peered":"2024-09-21T07:26:15.763516+0000","last_unstale":"2024-09-21T07:26:23.217510+0000","last_undegraded":"2024-09-21T07:26:23.217510+0000","last_fullsized":"2024-09-21T07:26:23.217510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:48.556036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217467+0000","last_change":"2024-09-21T07:26:19.784787+0000","last_active":"2024-09-21T07:26:23.217467+0000","last_peered":"2024-09-21T07:26:23.217467+0000","last_clean":"2024-09-21T07:26:23.217467+0000","last_became_active":"2024-09-21T07:26:19.784262+0000","last_became_peered":"2024-09-21T07:26:19.784262+0000","last_unstale":"2024-09-21T07:26:23.217467+0000","last_undegraded":"2024-09-21T07:26:23.217467+0000","last_fullsized":"2024-09-21T07:26:23.217467+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:18.757962+0000","last_clean_scrub_stamp":"2024-09-21T07:26:18.757962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:00:47.369615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790131+0000","last_change":"2024-09-21T07:26:15.762178+0000","last_active":"2024-09-21T07:26:21.790131+0000","last_peered":"2024-09-21T07:26:21.790131+0000","last_clean":"2024-09-21T07:26:21.790131+0000","last_became_active":"2024-09-21T07:26:15.761535+0000","last_became_peered":"2024-09-21T07:26:15.761535+0000","last_unstale":"2024-09-21T07:26:21.790131+0000","last_undegraded":"2024-09-21T07:26:21.790131+0000","last_fullsized":"2024-09-21T07:26:21.790131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:00.235854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789216+0000","last_change":"2024-09-21T07:26:15.760862+0000","last_active":"2024-09-21T07:26:21.789216+0000","last_peered":"2024-09-21T07:26:21.789216+0000","last_clean":"2024-09-21T07:26:21.789216+0000","last_became_active":"2024-09-21T07:26:15.760547+0000","last_became_peered":"2024-09-21T07:26:15.760547+0000","last_unstale":"2024-09-21T07:26:21.789216+0000","last_undegraded":"2024-09-21T07:26:21.789216+0000","last_fullsized":"2024-09-21T07:26:21.789216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:49.131007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900702+0000","last_change":"2024-09-21T07:26:15.773293+0000","last_active":"2024-09-21T07:26:22.900702+0000","last_peered":"2024-09-21T07:26:22.900702+0000","last_clean":"2024-09-21T07:26:22.900702+0000","last_became_active":"2024-09-21T07:26:15.772904+0000","last_became_peered":"2024-09-21T07:26:15.772904+0000","last_unstale":"2024-09-21T07:26:22.900702+0000","last_undegraded":"2024-09-21T07:26:22.900702+0000","last_fullsized":"2024-09-21T07:26:22.900702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:56.727494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756234+0000","last_change":"2024-09-21T07:26:15.764081+0000","last_active":"2024-09-21T07:26:22.756234+0000","last_peered":"2024-09-21T07:26:22.756234+0000","last_clean":"2024-09-21T07:26:22.756234+0000","last_became_active":"2024-09-21T07:26:15.763052+0000","last_became_peered":"2024-09-21T07:26:15.763052+0000","last_unstale":"2024-09-21T07:26:22.756234+0000","last_undegraded":"2024-09-21T07:26:22.756234+0000","last_fullsized":"2024-09-21T07:26:22.756234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218342+0000","last_change":"2024-09-21T07:26:15.766994+0000","last_active":"2024-09-21T07:26:23.218342+0000","last_peered":"2024-09-21T07:26:23.218342+0000","last_clean":"2024-09-21T07:26:23.218342+0000","last_became_active":"2024-09-21T07:26:15.766623+0000","last_became_peered":"2024-09-21T07:26:15.766623+0000","last_unstale":"2024-09-21T07:26:23.218342+0000","last_undegraded":"2024-09-21T07:26:23.218342+0000","last_fullsized":"2024-09-21T07:26:23.218342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:59.716664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218240+0000","last_change":"2024-09-21T07:26:15.758623+0000","last_active":"2024-09-21T07:26:23.218240+0000","last_peered":"2024-09-21T07:26:23.218240+0000","last_clean":"2024-09-21T07:26:23.218240+0000","last_became_active":"2024-09-21T07:26:15.758309+0000","last_became_peered":"2024-09-21T07:26:15.758309+0000","last_unstale":"2024-09-21T07:26:23.218240+0000","last_undegraded":"2024-09-21T07:26:23.218240+0000","last_fullsized":"2024-09-21T07:26:23.218240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.792006+0000","last_change":"2024-09-21T07:26:15.770009+0000","last_active":"2024-09-21T07:26:21.792006+0000","last_peered":"2024-09-21T07:26:21.792006+0000","last_clean":"2024-09-21T07:26:21.792006+0000","last_became_active":"2024-09-21T07:26:15.769549+0000","last_became_peered":"2024-09-21T07:26:15.769549+0000","last_unstale":"2024-09-21T07:26:21.792006+0000","last_undegraded":"2024-09-21T07:26:21.792006+0000","last_fullsized":"2024-09-21T07:26:21.792006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218223+0000","last_change":"2024-09-21T07:26:15.765956+0000","last_active":"2024-09-21T07:26:23.218223+0000","last_peered":"2024-09-21T07:26:23.218223+0000","last_clean":"2024-09-21T07:26:23.218223+0000","last_became_active":"2024-09-21T07:26:15.765561+0000","last_became_peered":"2024-09-21T07:26:15.765561+0000","last_unstale":"2024-09-21T07:26:23.218223+0000","last_undegraded":"2024-09-21T07:26:23.218223+0000","last_fullsized":"2024-09-21T07:26:23.218223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:35.944784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900539+0000","last_change":"2024-09-21T07:26:15.766091+0000","last_active":"2024-09-21T07:26:22.900539+0000","last_peered":"2024-09-21T07:26:22.900539+0000","last_clean":"2024-09-21T07:26:22.900539+0000","last_became_active":"2024-09-21T07:26:15.765771+0000","last_became_peered":"2024-09-21T07:26:15.765771+0000","last_unstale":"2024-09-21T07:26:22.900539+0000","last_undegraded":"2024-09-21T07:26:22.900539+0000","last_fullsized":"2024-09-21T07:26:22.900539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:15.463622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217240+0000","last_change":"2024-09-21T07:26:15.771905+0000","last_active":"2024-09-21T07:26:23.217240+0000","last_peered":"2024-09-21T07:26:23.217240+0000","last_clean":"2024-09-21T07:26:23.217240+0000","last_became_active":"2024-09-21T07:26:15.771525+0000","last_became_peered":"2024-09-21T07:26:15.771525+0000","last_unstale":"2024-09-21T07:26:23.217240+0000","last_undegraded":"2024-09-21T07:26:23.217240+0000","last_fullsized":"2024-09-21T07:26:23.217240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:25.988043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900725+0000","last_change":"2024-09-21T07:26:15.775865+0000","last_active":"2024-09-21T07:26:22.900725+0000","last_peered":"2024-09-21T07:26:22.900725+0000","last_clean":"2024-09-21T07:26:22.900725+0000","last_became_active":"2024-09-21T07:26:15.773173+0000","last_became_peered":"2024-09-21T07:26:15.773173+0000","last_unstale":"2024-09-21T07:26:22.900725+0000","last_undegraded":"2024-09-21T07:26:22.900725+0000","last_fullsized":"2024-09-21T07:26:22.900725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:33:42.898368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.756041+0000","last_change":"2024-09-21T07:26:15.766057+0000","last_active":"2024-09-21T07:26:22.756041+0000","last_peered":"2024-09-21T07:26:22.756041+0000","last_clean":"2024-09-21T07:26:22.756041+0000","last_became_active":"2024-09-21T07:26:15.765714+0000","last_became_peered":"2024-09-21T07:26:15.765714+0000","last_unstale":"2024-09-21T07:26:22.756041+0000","last_undegraded":"2024-09-21T07:26:22.756041+0000","last_fullsized":"2024-09-21T07:26:22.756041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790549+0000","last_change":"2024-09-21T07:26:15.774046+0000","last_active":"2024-09-21T07:26:21.790549+0000","last_peered":"2024-09-21T07:26:21.790549+0000","last_clean":"2024-09-21T07:26:21.790549+0000","last_became_active":"2024-09-21T07:26:15.773709+0000","last_became_peered":"2024-09-21T07:26:15.773709+0000","last_unstale":"2024-09-21T07:26:21.790549+0000","last_undegraded":"2024-09-21T07:26:21.790549+0000","last_fullsized":"2024-09-21T07:26:21.790549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:18.342308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791898+0000","last_change":"2024-09-21T07:26:15.760349+0000","last_active":"2024-09-21T07:26:21.791898+0000","last_peered":"2024-09-21T07:26:21.791898+0000","last_clean":"2024-09-21T07:26:21.791898+0000","last_became_active":"2024-09-21T07:26:15.759759+0000","last_became_peered":"2024-09-21T07:26:15.759759+0000","last_unstale":"2024-09-21T07:26:21.791898+0000","last_undegraded":"2024-09-21T07:26:21.791898+0000","last_fullsized":"2024-09-21T07:26:21.791898+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:33.433481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.789376+0000","last_change":"2024-09-21T07:26:15.767539+0000","last_active":"2024-09-21T07:26:21.789376+0000","last_peered":"2024-09-21T07:26:21.789376+0000","last_clean":"2024-09-21T07:26:21.789376+0000","last_became_active":"2024-09-21T07:26:15.767174+0000","last_became_peered":"2024-09-21T07:26:15.767174+0000","last_unstale":"2024-09-21T07:26:21.789376+0000","last_undegraded":"2024-09-21T07:26:21.789376+0000","last_fullsized":"2024-09-21T07:26:21.789376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:08.568390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.901369+0000","last_change":"2024-09-21T07:26:15.767788+0000","last_active":"2024-09-21T07:26:22.901369+0000","last_peered":"2024-09-21T07:26:22.901369+0000","last_clean":"2024-09-21T07:26:22.901369+0000","last_became_active":"2024-09-21T07:26:15.767459+0000","last_became_peered":"2024-09-21T07:26:15.767459+0000","last_unstale":"2024-09-21T07:26:22.901369+0000","last_undegraded":"2024-09-21T07:26:22.901369+0000","last_fullsized":"2024-09-21T07:26:22.901369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:31.318947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900393+0000","last_change":"2024-09-21T07:26:15.758591+0000","last_active":"2024-09-21T07:26:22.900393+0000","last_peered":"2024-09-21T07:26:22.900393+0000","last_clean":"2024-09-21T07:26:22.900393+0000","last_became_active":"2024-09-21T07:26:15.758248+0000","last_became_peered":"2024-09-21T07:26:15.758248+0000","last_unstale":"2024-09-21T07:26:22.900393+0000","last_undegraded":"2024-09-21T07:26:22.900393+0000","last_fullsized":"2024-09-21T07:26:22.900393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:36.622164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.216601+0000","last_change":"2024-09-21T07:26:15.763665+0000","last_active":"2024-09-21T07:26:23.216601+0000","last_peered":"2024-09-21T07:26:23.216601+0000","last_clean":"2024-09-21T07:26:23.216601+0000","last_became_active":"2024-09-21T07:26:15.763348+0000","last_became_peered":"2024-09-21T07:26:15.763348+0000","last_unstale":"2024-09-21T07:26:23.216601+0000","last_undegraded":"2024-09-21T07:26:23.216601+0000","last_fullsized":"2024-09-21T07:26:23.216601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:16.203476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755781+0000","last_change":"2024-09-21T07:26:15.757427+0000","last_active":"2024-09-21T07:26:22.755781+0000","last_peered":"2024-09-21T07:26:22.755781+0000","last_clean":"2024-09-21T07:26:22.755781+0000","last_became_active":"2024-09-21T07:26:15.757078+0000","last_became_peered":"2024-09-21T07:26:15.757078+0000","last_unstale":"2024-09-21T07:26:22.755781+0000","last_undegraded":"2024-09-21T07:26:22.755781+0000","last_fullsized":"2024-09-21T07:26:22.755781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:46.647829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217916+0000","last_change":"2024-09-21T07:26:15.767064+0000","last_active":"2024-09-21T07:26:23.217916+0000","last_peered":"2024-09-21T07:26:23.217916+0000","last_clean":"2024-09-21T07:26:23.217916+0000","last_became_active":"2024-09-21T07:26:15.766690+0000","last_became_peered":"2024-09-21T07:26:15.766690+0000","last_unstale":"2024-09-21T07:26:23.217916+0000","last_undegraded":"2024-09-21T07:26:23.217916+0000","last_fullsized":"2024-09-21T07:26:23.217916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:07.860716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217825+0000","last_change":"2024-09-21T07:26:15.775975+0000","last_active":"2024-09-21T07:26:23.217825+0000","last_peered":"2024-09-21T07:26:23.217825+0000","last_clean":"2024-09-21T07:26:23.217825+0000","last_became_active":"2024-09-21T07:26:15.775526+0000","last_became_peered":"2024-09-21T07:26:15.775526+0000","last_unstale":"2024-09-21T07:26:23.217825+0000","last_undegraded":"2024-09-21T07:26:23.217825+0000","last_fullsized":"2024-09-21T07:26:23.217825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:13.840733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791070+0000","last_change":"2024-09-21T07:26:15.776942+0000","last_active":"2024-09-21T07:26:21.791070+0000","last_peered":"2024-09-21T07:26:21.791070+0000","last_clean":"2024-09-21T07:26:21.791070+0000","last_became_active":"2024-09-21T07:26:15.776632+0000","last_became_peered":"2024-09-21T07:26:15.776632+0000","last_unstale":"2024-09-21T07:26:21.791070+0000","last_undegraded":"2024-09-21T07:26:21.791070+0000","last_fullsized":"2024-09-21T07:26:21.791070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:15.037813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791878+0000","last_change":"2024-09-21T07:26:15.766327+0000","last_active":"2024-09-21T07:26:21.791878+0000","last_peered":"2024-09-21T07:26:21.791878+0000","last_clean":"2024-09-21T07:26:21.791878+0000","last_became_active":"2024-09-21T07:26:15.765981+0000","last_became_peered":"2024-09-21T07:26:15.765981+0000","last_unstale":"2024-09-21T07:26:21.791878+0000","last_undegraded":"2024-09-21T07:26:21.791878+0000","last_fullsized":"2024-09-21T07:26:21.791878+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:44.617184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791919+0000","last_change":"2024-09-21T07:26:15.777412+0000","last_active":"2024-09-21T07:26:21.791919+0000","last_peered":"2024-09-21T07:26:21.791919+0000","last_clean":"2024-09-21T07:26:21.791919+0000","last_became_active":"2024-09-21T07:26:15.776925+0000","last_became_peered":"2024-09-21T07:26:15.776925+0000","last_unstale":"2024-09-21T07:26:21.791919+0000","last_undegraded":"2024-09-21T07:26:21.791919+0000","last_fullsized":"2024-09-21T07:26:21.791919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:38.270679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.218556+0000","last_change":"2024-09-21T07:26:15.777075+0000","last_active":"2024-09-21T07:26:23.218556+0000","last_peered":"2024-09-21T07:26:23.218556+0000","last_clean":"2024-09-21T07:26:23.218556+0000","last_became_active":"2024-09-21T07:26:15.776611+0000","last_became_peered":"2024-09-21T07:26:15.776611+0000","last_unstale":"2024-09-21T07:26:23.218556+0000","last_undegraded":"2024-09-21T07:26:23.218556+0000","last_fullsized":"2024-09-21T07:26:23.218556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:38.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.790866+0000","last_change":"2024-09-21T07:26:15.762408+0000","last_active":"2024-09-21T07:26:21.790866+0000","last_peered":"2024-09-21T07:26:21.790866+0000","last_clean":"2024-09-21T07:26:21.790866+0000","last_became_active":"2024-09-21T07:26:15.762014+0000","last_became_peered":"2024-09-21T07:26:15.762014+0000","last_unstale":"2024-09-21T07:26:21.790866+0000","last_undegraded":"2024-09-21T07:26:21.790866+0000","last_fullsized":"2024-09-21T07:26:21.790866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:09.186226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900571+0000","last_change":"2024-09-21T07:26:15.759759+0000","last_active":"2024-09-21T07:26:22.900571+0000","last_peered":"2024-09-21T07:26:22.900571+0000","last_clean":"2024-09-21T07:26:22.900571+0000","last_became_active":"2024-09-21T07:26:15.758871+0000","last_became_peered":"2024-09-21T07:26:15.758871+0000","last_unstale":"2024-09-21T07:26:22.900571+0000","last_undegraded":"2024-09-21T07:26:22.900571+0000","last_fullsized":"2024-09-21T07:26:22.900571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:23.666201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791461+0000","last_change":"2024-09-21T07:26:15.768245+0000","last_active":"2024-09-21T07:26:21.791461+0000","last_peered":"2024-09-21T07:26:21.791461+0000","last_clean":"2024-09-21T07:26:21.791461+0000","last_became_active":"2024-09-21T07:26:15.767901+0000","last_became_peered":"2024-09-21T07:26:15.767901+0000","last_unstale":"2024-09-21T07:26:21.791461+0000","last_undegraded":"2024-09-21T07:26:21.791461+0000","last_fullsized":"2024-09-21T07:26:21.791461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:52.823399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.900922+0000","last_change":"2024-09-21T07:26:15.772040+0000","last_active":"2024-09-21T07:26:22.900922+0000","last_peered":"2024-09-21T07:26:22.900922+0000","last_clean":"2024-09-21T07:26:22.900922+0000","last_became_active":"2024-09-21T07:26:15.771194+0000","last_became_peered":"2024-09-21T07:26:15.771194+0000","last_unstale":"2024-09-21T07:26:22.900922+0000","last_undegraded":"2024-09-21T07:26:22.900922+0000","last_fullsized":"2024-09-21T07:26:22.900922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:35.969051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:22.755834+0000","last_change":"2024-09-21T07:26:15.752787+0000","last_active":"2024-09-21T07:26:22.755834+0000","last_peered":"2024-09-21T07:26:22.755834+0000","last_clean":"2024-09-21T07:26:22.755834+0000","last_became_active":"2024-09-21T07:26:15.752457+0000","last_became_peered":"2024-09-21T07:26:15.752457+0000","last_unstale":"2024-09-21T07:26:22.755834+0000","last_undegraded":"2024-09-21T07:26:22.755834+0000","last_fullsized":"2024-09-21T07:26:22.755834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:44.608965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:23.217328+0000","last_change":"2024-09-21T07:26:15.754185+0000","last_active":"2024-09-21T07:26:23.217328+0000","last_peered":"2024-09-21T07:26:23.217328+0000","last_clean":"2024-09-21T07:26:23.217328+0000","last_became_active":"2024-09-21T07:26:15.753790+0000","last_became_peered":"2024-09-21T07:26:15.753790+0000","last_unstale":"2024-09-21T07:26:23.217328+0000","last_undegraded":"2024-09-21T07:26:23.217328+0000","last_fullsized":"2024-09-21T07:26:23.217328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:21.190898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-21T07:26:21.791398+0000","last_change":"2024-09-21T07:26:15.769408+0000","last_active":"2024-09-21T07:26:21.791398+0000","last_peered":"2024-09-21T07:26:21.791398+0000","last_clean":"2024-09-21T07:26:21.791398+0000","last_became_active":"2024-09-21T07:26:15.767688+0000","last_became_peered":"2024-09-21T07:26:15.767688+0000","last_unstale":"2024-09-21T07:26:21.791398+0000","last_undegraded":"2024-09-21T07:26:21.791398+0000","last_fullsized":"2024-09-21T07:26:21.791398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:14.721255+0000","last_clean_scrub_stamp":"2024-09-21T07:26:14.721255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:46.925801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":880640,"data_stored":704845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":286720,"data_stored":113985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":294912,"data_stored":114021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":113895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509452,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":114055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":888832,"data_stored":704901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":117737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:26:42.036 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:26:42.036 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T07:26:42.036 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T07:26:42.036 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.180+0000 7fa515932640 1 -- 172.21.15.105:0/1775399415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 msgr2=0x7fa5100b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.180+0000 7fa515932640 1 --2- 172.21.15.105:0/1775399415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5100b3cb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa51008d170 tx=0x7fa50002a8f0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 -- 172.21.15.105:0/1775399415 shutdown_connections 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 --2- 172.21.15.105:0/1775399415 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 0x7fa5100d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 --2- 172.21.15.105:0/1775399415 >> v2:172.21.15.112:3300/0 conn(0x7fa5100d4460 0x7fa5100d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 --2- 172.21.15.105:0/1775399415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5100b3cb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 -- 172.21.15.105:0/1775399415 >> 172.21.15.105:0/1775399415 conn(0x7fa51008bfb0 msgr2=0x7fa51009d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 -- 172.21.15.105:0/1775399415 shutdown_connections 2024-09-21T07:26:42.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.181+0000 7fa515932640 1 -- 172.21.15.105:0/1775399415 wait complete. 2024-09-21T07:26:42.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.182+0000 7fa515932640 1 Processor -- start 2024-09-21T07:26:42.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.183+0000 7fa515932640 1 -- start start 2024-09-21T07:26:42.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 0x7fa5101e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5100d4460 0x7fa5101e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100b4aa0 con 0x7fa5100d3020 2024-09-21T07:26:42.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5100b4920 con 0x7fa5100d4460 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa515932640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa5100d1910 con 0x7fa5100d0fc0 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60724/0 (socket says 172.21.15.105:60724) 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 -- 172.21.15.105:0/3188450288 learned_addr learned my addr 172.21.15.105:0/3188450288 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50effd640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 0x7fa5101e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50f7fe640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5100d4460 0x7fa5101e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 -- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 msgr2=0x7fa5101e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 0x7fa5101e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 -- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5100d4460 msgr2=0x7fa5101e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5100d4460 0x7fa5101e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 -- 172.21.15.105:0/3188450288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51015e770 con 0x7fa5100d3020 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa50e7fc640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d740 tx=0x7fa50400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.184+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504017070 con 0x7fa5100d3020 2024-09-21T07:26:42.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.185+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa504009ab0 con 0x7fa5100d3020 2024-09-21T07:26:42.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.185+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51015ea50 con 0x7fa5100d3020 2024-09-21T07:26:42.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.185+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504010460 con 0x7fa5100d3020 2024-09-21T07:26:42.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.185+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5101f5600 con 0x7fa5100d3020 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.186+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5100b3cb0 con 0x7fa5100d3020 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.186+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa50401a070 con 0x7fa5100d3020 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.187+0000 7fa4f7fff640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 0x7fa4fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.187+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa50409b120 con 0x7fa5100d3020 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.187+0000 7fa50effd640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 0x7fa4fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.190 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.188+0000 7fa50effd640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 0x7fa4fc07b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa500035000 tx=0x7fa500004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.191+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa504062b50 con 0x7fa5100d3020 2024-09-21T07:26:42.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:26:42.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:26:42.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.406+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa5101eb310 con 0x7fa5100d3020 2024-09-21T07:26:42.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.407+0000 7fa4f7fff640 1 -- 172.21.15.105:0/3188450288 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa504067580 con 0x7fa5100d3020 2024-09-21T07:26:42.409 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:42.409 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 msgr2=0x7fa4fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 0x7fa4fc07b0d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa500035000 tx=0x7fa500004ab0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 msgr2=0x7fa5101f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa50400d740 tx=0x7fa50400dc00 comp rx=0 tx=0).stop 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 shutdown_connections 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa4fc078bc0 0x7fa4fc07b0d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5100d4460 0x7fa5101e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5100d3020 0x7fa5101f50c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 --2- 172.21.15.105:0/3188450288 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5100d0fc0 0x7fa5101e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 >> 172.21.15.105:0/3188450288 conn(0x7fa51008bfb0 msgr2=0x7fa510161d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 shutdown_connections 2024-09-21T07:26:42.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.409+0000 7fa515932640 1 -- 172.21.15.105:0/3188450288 wait complete. 2024-09-21T07:26:42.422 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T07:26:42.422 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 -- 172.21.15.105:0/3897038280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa500b6850 msgr2=0x7ffa500cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 --2- 172.21.15.105:0/3897038280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa500b6850 0x7ffa500cf760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffa500a25a0 tx=0x7ffa4802aac0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 -- 172.21.15.105:0/3897038280 shutdown_connections 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 --2- 172.21.15.105:0/3897038280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa500b6850 0x7ffa500cf760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 --2- 172.21.15.105:0/3897038280 >> v2:172.21.15.112:3300/0 conn(0x7ffa5008c3c0 0x7ffa5006f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 -- 172.21.15.105:0/3897038280 >> 172.21.15.105:0/3897038280 conn(0x7ffa5008bfb0 msgr2=0x7ffa500d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 -- 172.21.15.105:0/3897038280 shutdown_connections 2024-09-21T07:26:42.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.601+0000 7ffa54fad640 1 -- 172.21.15.105:0/3897038280 wait complete. 2024-09-21T07:26:42.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.603+0000 7ffa54fad640 1 Processor -- start 2024-09-21T07:26:42.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.603+0000 7ffa54fad640 1 -- start start 2024-09-21T07:26:42.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 0x7ffa501f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 0x7ffa500c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500b3ad0 con 0x7ffa5008c3c0 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa500b3950 con 0x7ffa500b6440 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa54fad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ffa50162c70 con 0x7ffa500b6850 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60744/0 (socket says 172.21.15.105:60744) 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 -- 172.21.15.105:0/2455340091 learned_addr learned my addr 172.21.15.105:0/2455340091 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4dd9b640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 0x7ffa501f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4ed9d640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 0x7ffa500c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 -- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 msgr2=0x7ffa500c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 0x7ffa500c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 -- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 msgr2=0x7ffa501f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 0x7ffa501f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4e59c640 1 -- 172.21.15.105:0/2455340091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa500c8230 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4ed9d640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 0x7ffa500c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.604+0000 7ffa4dd9b640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 0x7ffa501f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa4e59c640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000d700 tx=0x7ffa4000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40017070 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa40005310 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa500c8510 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40009820 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.605+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa500c8ad0 con 0x7ffa5008c3c0 2024-09-21T07:26:42.609 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.606+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5006f7f0 con 0x7ffa5008c3c0 2024-09-21T07:26:42.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.607+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa400099c0 con 0x7ffa5008c3c0 2024-09-21T07:26:42.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.607+0000 7ffa3effd640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 0x7ffa3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.607+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ffa4009b0f0 con 0x7ffa5008c3c0 2024-09-21T07:26:42.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.607+0000 7ffa4dd9b640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 0x7ffa3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.608+0000 7ffa4dd9b640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 0x7ffa3007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006eec0 tx=0x7ffa44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.612+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa40063310 con 0x7ffa5008c3c0 2024-09-21T07:26:42.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.821+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa500b6d80 con 0x7ffa5008c3c0 2024-09-21T07:26:42.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.822+0000 7ffa3effd640 1 -- 172.21.15.105:0/2455340091 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7ffa40062c10 con 0x7ffa5008c3c0 2024-09-21T07:26:42.824 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:42.824 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:42.824 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 msgr2=0x7ffa3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 0x7ffa3007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa5006eec0 tx=0x7ffa44009040 comp rx=0 tx=0).stop 2024-09-21T07:26:42.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 msgr2=0x7ffa50161fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000d700 tx=0x7ffa4000dbc0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 shutdown_connections 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ffa30078b90 0x7ffa3007b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa500b6440 0x7ffa501f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ffa500b6850 0x7ffa500c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 --2- 172.21.15.105:0/2455340091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffa5008c3c0 0x7ffa50161fd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 >> 172.21.15.105:0/2455340091 conn(0x7ffa5008bfb0 msgr2=0x7ffa500b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 shutdown_connections 2024-09-21T07:26:42.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.824+0000 7ffa54fad640 1 -- 172.21.15.105:0/2455340091 wait complete. 2024-09-21T07:26:42.838 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:42.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 -- 172.21.15.105:0/384057574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cff70 msgr2=0x7fbd0c0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/384057574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cff70 0x7fbd0c0d0380 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009830 tx=0x7fbd0802a510 comp rx=0 tx=0).stop 2024-09-21T07:26:42.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 -- 172.21.15.105:0/384057574 shutdown_connections 2024-09-21T07:26:42.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/384057574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cd090 0x7fbd0c0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/384057574 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cff70 0x7fbd0c0d0380 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 -- 172.21.15.105:0/384057574 >> 172.21.15.105:0/384057574 conn(0x7fbd0c0cee20 msgr2=0x7fbd0c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:42.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 -- 172.21.15.105:0/384057574 shutdown_connections 2024-09-21T07:26:42.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.949+0000 7fbd13bcc640 1 -- 172.21.15.105:0/384057574 wait complete. 2024-09-21T07:26:42.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.951+0000 7fbd13bcc640 1 Processor -- start 2024-09-21T07:26:42.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.951+0000 7fbd13bcc640 1 -- start start 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd0c0cd9e0 0x7fbd0c09ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cff70 0x7fbd0c1e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c0b7950 con 0x7fbd0c0cd090 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0c0b77d0 con 0x7fbd0c0cd9e0 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd13bcc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fbd0c0cddf0 con 0x7fbd0c0cff70 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60762/0 (socket says 172.21.15.105:60762) 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 -- 172.21.15.105:0/784343301 learned_addr learned my addr 172.21.15.105:0/784343301 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd12169640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cff70 0x7fbd0c1e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 -- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cff70 msgr2=0x7fbd0c1e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11167640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd0c0cd9e0 0x7fbd0c09ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cff70 0x7fbd0c1e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 -- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd0c0cd9e0 msgr2=0x7fbd0c09ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd0c0cd9e0 0x7fbd0c09ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 -- 172.21.15.105:0/784343301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0c09fa40 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.952+0000 7fbd11968640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009950 tx=0x7fbd08033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.953+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08005c90 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.953+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd08031e50 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.953+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0c09fd20 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.953+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08005260 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.953+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0c1f3810 con 0x7fbd0c0cd090 2024-09-21T07:26:42.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.954+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0c1df9c0 con 0x7fbd0c0cd090 2024-09-21T07:26:42.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.954+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd08005400 con 0x7fbd0c0cd090 2024-09-21T07:26:42.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.955+0000 7fbd027fc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 0x7fbcf007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:42.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.956+0000 7fbd11167640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 0x7fbcf007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:42.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.956+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbd080bc620 con 0x7fbd0c0cd090 2024-09-21T07:26:42.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.956+0000 7fbd11167640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 0x7fbcf007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c09ed20 tx=0x7fbcfc005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:42.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:42.959+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd0803f030 con 0x7fbd0c0cd090 2024-09-21T07:26:43.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.175+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd0c09bba0 con 0x7fbd0c0cd090 2024-09-21T07:26:43.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.175+0000 7fbd027fc640 1 -- 172.21.15.105:0/784343301 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7fbd08084710 con 0x7fbd0c0cd090 2024-09-21T07:26:43.177 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:43.178 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:43.178 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 msgr2=0x7fbcf007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 0x7fbcf007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c09ed20 tx=0x7fbcfc005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 msgr2=0x7fbd0c1e03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009950 tx=0x7fbd08033c20 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 shutdown_connections 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbcf0078bc0 0x7fbcf007b0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd0c0cd9e0 0x7fbd0c09ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd0c0cff70 0x7fbd0c1e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 --2- 172.21.15.105:0/784343301 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd0c0cd090 0x7fbd0c1e03d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.177+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 >> 172.21.15.105:0/784343301 conn(0x7fbd0c0cee20 msgr2=0x7fbd0c1df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:43.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.178+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 shutdown_connections 2024-09-21T07:26:43.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.178+0000 7fbd13bcc640 1 -- 172.21.15.105:0/784343301 wait complete. 2024-09-21T07:26:43.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:43.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 -- 172.21.15.105:0/2787428531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80d4460 msgr2=0x7f85c80d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 --2- 172.21.15.105:0/2787428531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80d4460 0x7f85c80d4870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009830 tx=0x7f85bc02a530 comp rx=0 tx=0).stop 2024-09-21T07:26:43.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 -- 172.21.15.105:0/2787428531 shutdown_connections 2024-09-21T07:26:43.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 --2- 172.21.15.105:0/2787428531 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80d4460 0x7f85c80d4870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 -- 172.21.15.105:0/2787428531 >> 172.21.15.105:0/2787428531 conn(0x7f85c808bfb0 msgr2=0x7f85c809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:43.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 -- 172.21.15.105:0/2787428531 shutdown_connections 2024-09-21T07:26:43.305 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.302+0000 7f85cd2de640 1 -- 172.21.15.105:0/2787428531 wait complete. 2024-09-21T07:26:43.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.304+0000 7f85cd2de640 1 Processor -- start 2024-09-21T07:26:43.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.304+0000 7f85cd2de640 1 -- start start 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c80d3020 0x7f85c81f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 0x7f85c80d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f85c80cfcd0 con 0x7f85c80cf1b0 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85c80b4b30 con 0x7f85c80d4460 2024-09-21T07:26:43.308 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85cd2de640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f85c80cfe50 con 0x7f85c80d3020 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85c6ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60780/0 (socket says 172.21.15.105:60780) 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.305+0000 7f85c6ffd640 1 -- 172.21.15.105:0/514240474 learned_addr learned my addr 172.21.15.105:0/514240474 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c77fe640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 0x7f85c80d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c67fc640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c80d3020 0x7f85c81f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 -- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c80d3020 msgr2=0x7f85c81f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c80d3020 0x7f85c81f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 -- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 msgr2=0x7f85c80d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 0x7f85c80d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 -- 172.21.15.105:0/514240474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c81e8510 con 0x7f85c80cf1b0 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c77fe640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 0x7f85c80d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85c6ffd640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85b800c770 tx=0x7f85b800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:43.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8017070 con 0x7f85c80cf1b0 2024-09-21T07:26:43.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85b8005070 con 0x7f85c80cf1b0 2024-09-21T07:26:43.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c81e7750 con 0x7f85c80cf1b0 2024-09-21T07:26:43.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8005460 con 0x7f85c80cf1b0 2024-09-21T07:26:43.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.306+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81e7d10 con 0x7f85c80cf1b0 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.307+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c80d02d0 con 0x7f85c80cf1b0 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.308+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85b801a070 con 0x7f85c80cf1b0 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.308+0000 7f85af7fe640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 0x7f85b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.309+0000 7f85c67fc640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 0x7f85b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.309+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f85b809a6a0 con 0x7f85c80cf1b0 2024-09-21T07:26:43.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.309+0000 7f85c67fc640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 0x7f85b407b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85bc034000 tx=0x7f85bc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:43.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.313+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85b80627b0 con 0x7f85c80cf1b0 2024-09-21T07:26:43.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.491+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85c80c7e60 con 0x7f85c80cf1b0 2024-09-21T07:26:43.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.492+0000 7f85af7fe640 1 -- 172.21.15.105:0/514240474 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f85b80620b0 con 0x7f85c80cf1b0 2024-09-21T07:26:43.494 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:43.494 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:43.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 msgr2=0x7f85b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 0x7f85b407b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85bc034000 tx=0x7f85bc004730 comp rx=0 tx=0).stop 2024-09-21T07:26:43.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 msgr2=0x7f85c80b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f85b800c770 tx=0x7f85b800cc30 comp rx=0 tx=0).stop 2024-09-21T07:26:43.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 shutdown_connections 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f85b4078b90 0x7f85b407b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85c80d4460 0x7f85c80d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f85c80d3020 0x7f85c81f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 --2- 172.21.15.105:0/514240474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f85c80cf1b0 0x7f85c80b2830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 >> 172.21.15.105:0/514240474 conn(0x7f85c808bfb0 msgr2=0x7f85c8158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 shutdown_connections 2024-09-21T07:26:43.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.494+0000 7f85cd2de640 1 -- 172.21.15.105:0/514240474 wait complete. 2024-09-21T07:26:43.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 -- 172.21.15.105:0/551491050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 msgr2=0x7f4f3c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 --2- 172.21.15.105:0/551491050 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f3c003e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009b50 tx=0x7f4f3c005830 comp rx=0 tx=0).stop 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f5592d640 1 -- 172.21.15.105:0/551491050 reap_dead start 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 -- 172.21.15.105:0/551491050 shutdown_connections 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 -- 172.21.15.105:0/551491050 >> 172.21.15.105:0/551491050 conn(0x7f4f5009aa20 msgr2=0x7f4f500b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:43.620 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 -- 172.21.15.105:0/551491050 shutdown_connections 2024-09-21T07:26:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.618+0000 7f4f57390640 1 -- 172.21.15.105:0/551491050 wait complete. 2024-09-21T07:26:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.620+0000 7f4f57390640 1 Processor -- start 2024-09-21T07:26:43.621 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.620+0000 7f4f57390640 1 -- start start 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5009ec60 0x7f4f501f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f501f92b0 0x7f4f5014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500b6440 con 0x7f4f3c003a00 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500b62c0 con 0x7f4f501f92b0 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f57390640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f4f500b65c0 con 0x7f4f5009ec60 2024-09-21T07:26:43.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60788/0 (socket says 172.21.15.105:60788) 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 -- 172.21.15.105:0/4175680311 learned_addr learned my addr 172.21.15.105:0/4175680311 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5592d640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f501f92b0 0x7f4f5014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5492b640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5009ec60 0x7f4f501f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 -- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5009ec60 msgr2=0x7f4f501f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5009ec60 0x7f4f501f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 -- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f501f92b0 msgr2=0x7f4f5014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f501f92b0 0x7f4f5014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.621+0000 7f4f5512c640 1 -- 172.21.15.105:0/4175680311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5015b5b0 con 0x7f4f3c003a00 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f5512c640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c036040 tx=0x7f4f4c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c038050 con 0x7f4f3c003a00 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f4c039040 con 0x7f4f3c003a00 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5015a7f0 con 0x7f4f3c003a00 2024-09-21T07:26:43.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c033c50 con 0x7f4f3c003a00 2024-09-21T07:26:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.622+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5015adb0 con 0x7f4f3c003a00 2024-09-21T07:26:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.623+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f500ae990 con 0x7f4f3c003a00 2024-09-21T07:26:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.624+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f4c046020 con 0x7f4f3c003a00 2024-09-21T07:26:43.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.624+0000 7f4f45ffb640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 0x7f4f3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.624+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4f4c0bbac0 con 0x7f4f3c003a00 2024-09-21T07:26:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.624+0000 7f4f5492b640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 0x7f4f3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:43.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.625+0000 7f4f5492b640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 0x7f4f3407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f501f8aa0 tx=0x7f4f40001cd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:43.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.629+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f4c04d020 con 0x7f4f3c003a00 2024-09-21T07:26:43.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.842+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f50071220 con 0x7f4f3c003a00 2024-09-21T07:26:43.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.843+0000 7f4f45ffb640 1 -- 172.21.15.105:0/4175680311 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f4f4c083580 con 0x7f4f3c003a00 2024-09-21T07:26:43.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:43.845 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:43.845 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:43.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 msgr2=0x7f4f3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 0x7f4f3407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f501f8aa0 tx=0x7f4f40001cd0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 msgr2=0x7f4f5009e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:43.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c036040 tx=0x7f4f4c033670 comp rx=0 tx=0).stop 2024-09-21T07:26:43.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 shutdown_connections 2024-09-21T07:26:43.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f4f34078b90 0x7f4f3407b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4f501f92b0 0x7f4f5014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4f5009ec60 0x7f4f501f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 --2- 172.21.15.105:0/4175680311 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4f3c003a00 0x7f4f5009e720 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:43.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 >> 172.21.15.105:0/4175680311 conn(0x7f4f5009aa20 msgr2=0x7f4f500b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:43.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 shutdown_connections 2024-09-21T07:26:43.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:43.845+0000 7f4f57390640 1 -- 172.21.15.105:0/4175680311 wait complete. 2024-09-21T07:26:44.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:26:15.095198+0000', 'modified': '2024-09-21T07:26:35.182301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4434': {'gid': 4434, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6836/3289542241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3289542241}, {'type': 'v1', 'addr': '172.21.15.112:6836', 'nonce': 3289542241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4434, 'qdb_cluster': [4434]} 2024-09-21T07:26:44.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:26:44.264 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:26:44.380 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 -- 172.21.15.105:0/3487454788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 msgr2=0x7fb85000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/3487454788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb85000b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb868074670 tx=0x7fb850005830 comp rx=0 tx=0).stop 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 -- 172.21.15.105:0/3487454788 shutdown_connections 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/3487454788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb85000b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 -- 172.21.15.105:0/3487454788 >> 172.21.15.105:0/3487454788 conn(0x7fb8680cee20 msgr2=0x7fb8680cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 -- 172.21.15.105:0/3487454788 shutdown_connections 2024-09-21T07:26:44.381 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.378+0000 7fb86e4a9640 1 -- 172.21.15.105:0/3487454788 wait complete. 2024-09-21T07:26:44.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.380+0000 7fb86e4a9640 1 Processor -- start 2024-09-21T07:26:44.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.380+0000 7fb86e4a9640 1 -- start start 2024-09-21T07:26:44.384 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 0x7fb8681f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8681f9250 0x7fb86806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680b7950 con 0x7fb850005350 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680b77d0 con 0x7fb8681df640 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb86809ba30 con 0x7fb8681f9250 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60798/0 (socket says 172.21.15.105:60798) 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 -- 172.21.15.105:0/2734244077 learned_addr learned my addr 172.21.15.105:0/2734244077 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86ca46640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8681f9250 0x7fb86806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb8677fe640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 0x7fb8681f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 -- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8681f9250 msgr2=0x7fb86806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8681f9250 0x7fb86806e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 -- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 msgr2=0x7fb8681f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 0x7fb8681f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 -- 172.21.15.105:0/2734244077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8681f9660 con 0x7fb850005350 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb8677fe640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 0x7fb8681f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb867fff640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb858001be0 tx=0x7fb8580039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb864ff9640 1 -- 172.21.15.105:0/2734244077 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858015070 con 0x7fb850005350 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb864ff9640 1 -- 172.21.15.105:0/2734244077 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb858011840 con 0x7fb850005350 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86809ee70 con 0x7fb850005350 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb864ff9640 1 -- 172.21.15.105:0/2734244077 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858011c90 con 0x7fb850005350 2024-09-21T07:26:44.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.382+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86809f430 con 0x7fb850005350 2024-09-21T07:26:44.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.384+0000 7fb864ff9640 1 -- 172.21.15.105:0/2734244077 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb858018070 con 0x7fb850005350 2024-09-21T07:26:44.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.385+0000 7fb864ff9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 0x7fb85407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.385+0000 7fb864ff9640 1 -- 172.21.15.105:0/2734244077 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb85809bd50 con 0x7fb850005350 2024-09-21T07:26:44.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.385+0000 7fb8677fe640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 0x7fb85407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.387 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.385+0000 7fb8677fe640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 0x7fb85407b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb85c000c00 tx=0x7fb85c006180 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.386+0000 7fb875418740 1 Processor -- start 2024-09-21T07:26:44.388 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.386+0000 7fb875418740 1 -- start start 2024-09-21T07:26:44.389 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 0x5582dd3b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5582dd2ff820 con 0x5582dcfc4590 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582dd2eec50 con 0x5582dcfe5270 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x5582dd2f2a60 con 0x5582dcfbc3b0 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60812/0 (socket says 172.21.15.105:60812) 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 -- 172.21.15.105:0/3004363170 learned_addr learned my addr 172.21.15.105:0/3004363170 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb867fff640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 0x5582dd3b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 -- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 msgr2=0x5582dd3b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3b74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.390 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 -- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 msgr2=0x5582dd3b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 0x5582dd3b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 -- 172.21.15.105:0/3004363170 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582dd050850 con 0x5582dcfc4590 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb867fff640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 0x5582dd3b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.387+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb860009f90 tx=0x7fb860004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5b248036096f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb84dffb640 1 -- 172.21.15.105:0/3004363170 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860038040 con 0x5582dcfc4590 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb84dffb640 1 -- 172.21.15.105:0/3004363170 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb860044070 con 0x5582dcfc4590 2024-09-21T07:26:44.391 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb84dffb640 1 -- 172.21.15.105:0/3004363170 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860033660 con 0x5582dcfc4590 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 -- 172.21.15.105:0/3004363170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 msgr2=0x5582dd3d43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb860009f90 tx=0x7fb860004ab0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 -- 172.21.15.105:0/3004363170 shutdown_connections 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfc4590 0x5582dd3d43e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 --2- 172.21.15.105:0/3004363170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfe5270 0x5582dd3b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 -- 172.21.15.105:0/3004363170 >> 172.21.15.105:0/3004363170 conn(0x5582dcfd93c0 msgr2=0x5582dd38d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.392 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.388+0000 7fb875418740 1 -- 172.21.15.105:0/3004363170 shutdown_connections 2024-09-21T07:26:44.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.389+0000 7fb875418740 1 -- 172.21.15.105:0/3004363170 wait complete. 2024-09-21T07:26:44.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.389+0000 7fb875418740 1 -- start start 2024-09-21T07:26:44.393 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.390+0000 7fb875418740 1 Processor -- start 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfe5270 0x5582dd3a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5582dd05f620 con 0x5582dcfbc3b0 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582dd2945f0 con 0x5582dcfc4590 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x5582dd3cb150 con 0x5582dcfe5270 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60824/0 (socket says 172.21.15.105:60824) 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 -- 172.21.15.105:0/3799981189 learned_addr learned my addr 172.21.15.105:0/3799981189 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb875418740 10 client.0 _ll_register_callbacks cb 0x5582dd2980a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfe5270 0x5582dd3a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 -- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfe5270 msgr2=0x5582dd3a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfe5270 0x5582dd3a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 -- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 msgr2=0x5582dd3a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 -- 172.21.15.105:0/3799981189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582dd0c76b0 con 0x5582dcfbc3b0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb867fff640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8580609d0 tx=0x7fb858063cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70130b08a4284529 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.391+0000 7fb846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:26:44.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85801f040 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb85806a450 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb860044070 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb858068240 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 10 client.4842 fetch_fsmap learned FSMap version 9 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5582dd3ccce0 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb860044070 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fb85805f0b0 con 0x5582dcfbc3b0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 10 client.4842 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 10 client.4842 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 20 client.4842 populate_metadata read hostname 'smithi105' 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcfa3e40 0x5582dd3e44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 4 client.4842 mds_command: new command op to 4434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 --> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582dd301770 con 0x5582dcfa3e40 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.392+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcfa3e40 0x5582dd3e44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.393+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcfa3e40 0x5582dd3e44f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a291ae8e0a71d2fe server_cookie=c28938d473c1be1 in_seq=0 out_seq=0 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.393+0000 7fb846ffd640 10 client.4842 ms_handle_connect on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.393+0000 7fb846ffd640 1 -- 172.21.15.105:0/3799981189 <== mds.0 v2:172.21.15.112:6834/3289542241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582dd301770 con 0x5582dcfa3e40 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.393+0000 7fb846ffd640 10 client.4842 handle_command_reply: tid=0 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 client.4842 shutdown 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 msgr2=0x5582dd3cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8580609d0 tx=0x7fb858063cb0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 shutdown_connections 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcfa3e40 0x5582dd3e44f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfe5270 0x5582dd3a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb846ffd640 0 client.4842 ms_handle_reset on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 --2- 172.21.15.105:0/3799981189 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfbc3b0 0x5582dd3cc370 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 >> 172.21.15.105:0/3799981189 conn(0x5582dcfd93c0 msgr2=0x5582dd38d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 shutdown_connections 2024-09-21T07:26:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 1 -- 172.21.15.105:0/3799981189 wait complete. 2024-09-21T07:26:44.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.394+0000 7fb875418740 20 client.4842 trim_cache size 0 max 16384 2024-09-21T07:26:44.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.412+0000 7fb875418740 1 Processor -- start 2024-09-21T07:26:44.414 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.412+0000 7fb875418740 1 -- start start 2024-09-21T07:26:44.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfbc3b0 0x5582dd3a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 0x5582dd3a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb868202b40 con 0x5582dcfa3e40 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8681f6360 con 0x5582dcfbc3b0 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.413+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb8682024b0 con 0x5582dcfc4590 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60840/0 (socket says 172.21.15.105:60840) 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 -- 172.21.15.105:0/3445783279 learned_addr learned my addr 172.21.15.105:0/3445783279 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb8677fe640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfbc3b0 0x5582dd3a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 0x5582dd3a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 -- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 msgr2=0x5582dd3a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 0x5582dd3a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 -- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfbc3b0 msgr2=0x5582dd3a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfbc3b0 0x5582dd3a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 -- 172.21.15.105:0/3445783279 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582dd3d3890 con 0x5582dcfa3e40 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb86ca46640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 0x5582dd3a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb867fff640 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb85809c070 tx=0x7fb858011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15142975e90bb562 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb84d7fa640 1 -- 172.21.15.105:0/3445783279 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85801f040 con 0x5582dcfa3e40 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb84d7fa640 1 -- 172.21.15.105:0/3445783279 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb85806d0f0 con 0x5582dcfa3e40 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.414+0000 7fb84d7fa640 1 -- 172.21.15.105:0/3445783279 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858089020 con 0x5582dcfa3e40 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 -- 172.21.15.105:0/3445783279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 msgr2=0x5582dd3b70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb85809c070 tx=0x7fb858011ec0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 -- 172.21.15.105:0/3445783279 shutdown_connections 2024-09-21T07:26:44.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfbc3b0 0x5582dd3a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfc4590 0x5582dd3a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 --2- 172.21.15.105:0/3445783279 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3b70f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 -- 172.21.15.105:0/3445783279 >> 172.21.15.105:0/3445783279 conn(0x5582dcfd93c0 msgr2=0x5582dd38d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 -- 172.21.15.105:0/3445783279 shutdown_connections 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.415+0000 7fb875418740 1 -- 172.21.15.105:0/3445783279 wait complete. 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.416+0000 7fb875418740 1 -- start start 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.416+0000 7fb875418740 1 Processor -- start 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5582dd3c9f20 con 0x5582dcfa3e40 2024-09-21T07:26:44.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5582dd3a2b40 con 0x5582dcfc4590 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x5582dd3ca540 con 0x5582dcfbc3b0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb875418740 10 client.0 _ll_register_callbacks cb 0x7fb858062ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb867fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60846/0 (socket says 172.21.15.105:60846) 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb867fff640 1 -- 172.21.15.105:0/974562049 learned_addr learned my addr 172.21.15.105:0/974562049 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb86ca46640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.417+0000 7fb8677fe640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 -- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 msgr2=0x5582dd3c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 -- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 msgr2=0x5582dd3a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 -- 172.21.15.105:0/974562049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582dd3f4910 con 0x5582dcfa3e40 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb867fff640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8580636a0 tx=0x7fb858099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c611a39249204a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:44.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85801f070 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb85806a450 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85806d0f0 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fb858089040 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 10 client.4848 fetch_fsmap learned FSMap version 9 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5582dd3f4910 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb858089e50 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fb8580793d0 con 0x5582dcfa3e40 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 10 client.4848 fetch_fsmap finished waiting for FSMap version 9 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 10 client.4848 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 20 client.4848 populate_metadata read hostname 'smithi105' 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcf79280 0x5582dd3a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 4 client.4848 mds_command: new command op to 4434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:26:44.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.418+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 --> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] -- command(tid 0: {"prefix": "status"}) -- 0x5582dd2fd220 con 0x5582dcf79280 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb8677fe640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcf79280 0x5582dd3a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb846ffd640 10 client.4848 ms_handle_connect on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb8677fe640 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcf79280 0x5582dd3a3b20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1295ca80094dda server_cookie=64de860686e9c117 in_seq=0 out_seq=0 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb846ffd640 1 -- 172.21.15.105:0/974562049 <== mds.0 v2:172.21.15.112:6834/3289542241 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5582dd2fd220 con 0x5582dcf79280 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb846ffd640 10 client.4848 handle_command_reply: tid=0 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 client.4848 shutdown 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 msgr2=0x5582dd3caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8580636a0 tx=0x7fb858099fc0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 shutdown_connections 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:6834/3289542241,v1:172.21.15.112:6836/3289542241] conn(0x5582dcf79280 0x5582dd3a3b20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5582dcfc4590 0x5582dd3a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5582dcfbc3b0 0x5582dd3c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 --2- 172.21.15.105:0/974562049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582dcfa3e40 0x5582dd3caf80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb846ffd640 0 client.4848 ms_handle_reset on v2:172.21.15.112:6834/3289542241 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.419+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 >> 172.21.15.105:0/974562049 conn(0x5582dcfd93c0 msgr2=0x5582dd38cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 shutdown_connections 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb875418740 1 -- 172.21.15.105:0/974562049 wait complete. 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb875418740 20 client.4848 trim_cache size 0 max 16384 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15ee7004-a3e2-41ce-b1eb-6358972a1ae2", 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 4434, 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-21T07:26:44.428 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 28.664056505000001, 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 9, 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 27, 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 27, 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 34.703605572000001 2024-09-21T07:26:44.429 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 msgr2=0x7fb85407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 0x7fb85407b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb85c000c00 tx=0x7fb85c006180 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 msgr2=0x7fb86809e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb858001be0 tx=0x7fb8580039c0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 shutdown_connections 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb854078b90 0x7fb85407b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8681df640 0x7fb8681f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8681f9250 0x7fb86806e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 --2- 172.21.15.105:0/2734244077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb850005350 0x7fb86809e3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 >> 172.21.15.105:0/2734244077 conn(0x7fb8680cee20 msgr2=0x7fb8680b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 shutdown_connections 2024-09-21T07:26:44.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:44.420+0000 7fb86e4a9640 1 -- 172.21.15.105:0/2734244077 wait complete. 2024-09-21T07:26:44.433 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T07:26:44.518 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T07:26:44.519 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T07:26:44.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.519 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:26:44.535 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-21T07:26:44.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.536 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-21T07:26:44.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.601 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-21T07:26:44.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.699 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:26:44.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.715 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-21T07:26:44.782 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-09-21T07:26:44.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:26:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.783 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:26:44.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T07:26:44.874 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T07:26:44.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:26:44.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.876 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:26:44.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:44.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T07:26:45.004 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T07:26:45.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:26:45.006 INFO:tasks.kclient:client.0 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v2'} 2024-09-21T07:26:45.006 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:26:45.006 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:26:45.007 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 -- 172.21.15.105:0/2341689315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500b02c0 msgr2=0x7f8c5015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 --2- 172.21.15.105:0/2341689315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500b02c0 0x7f8c5015a120 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8c500cf560 tx=0x7f8c4002aa30 comp rx=0 tx=0).stop 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 -- 172.21.15.105:0/2341689315 shutdown_connections 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 --2- 172.21.15.105:0/2341689315 >> v2:172.21.15.112:3300/0 conn(0x7f8c5006df10 0x7f8c5015d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 --2- 172.21.15.105:0/2341689315 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c500b02c0 0x7f8c5015a120 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 -- 172.21.15.105:0/2341689315 >> 172.21.15.105:0/2341689315 conn(0x7f8c500d1270 msgr2=0x7f8c50073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 -- 172.21.15.105:0/2341689315 shutdown_connections 2024-09-21T07:26:45.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.117+0000 7f8c54920640 1 -- 172.21.15.105:0/2341689315 wait complete. 2024-09-21T07:26:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.119+0000 7f8c54920640 1 Processor -- start 2024-09-21T07:26:45.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.119+0000 7f8c54920640 1 -- start start 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c50073ea0 0x7f8c500ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 0x7f8c500addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b3e50 con 0x7f8c5006df10 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b3cd0 con 0x7f8c500b02c0 2024-09-21T07:26:45.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c54920640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8c500b3fd0 con 0x7f8c50073ea0 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c4e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60862/0 (socket says 172.21.15.105:60862) 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c4e59c640 1 -- 172.21.15.105:0/2659951329 learned_addr learned my addr 172.21.15.105:0/2659951329 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c4dd9b640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c50073ea0 0x7f8c500ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.120+0000 7f8c4ed9d640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 0x7f8c500addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 -- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c50073ea0 msgr2=0x7f8c500ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c50073ea0 0x7f8c500ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 -- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 msgr2=0x7f8c500addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 0x7f8c500addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 -- 172.21.15.105:0/2659951329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500ae940 con 0x7f8c5006df10 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4ed9d640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 0x7f8c500addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c4e59c640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c4002b020 tx=0x7f8c40003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40033050 con 0x7f8c5006df10 2024-09-21T07:26:45.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c400044f0 con 0x7f8c5006df10 2024-09-21T07:26:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c500aebb0 con 0x7f8c5006df10 2024-09-21T07:26:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40038450 con 0x7f8c5006df10 2024-09-21T07:26:45.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.121+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c500af960 con 0x7f8c5006df10 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.122+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c500aedf0 con 0x7f8c5006df10 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.123+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c40036070 con 0x7f8c5006df10 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.123+0000 7f8c3effd640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 0x7f8c3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.123+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8c400bc570 con 0x7f8c5006df10 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.124+0000 7f8c4dd9b640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 0x7f8c3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:45.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.124+0000 7f8c4dd9b640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 0x7f8c3007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8c501f8990 tx=0x7f8c4400b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:45.129 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.128+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c500aedf0 con 0x7f8c5006df10 2024-09-21T07:26:45.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.341+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8c500afe60 con 0x7f8c5006df10 2024-09-21T07:26:45.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.341+0000 7f8c3effd640 1 -- 172.21.15.105:0/2659951329 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f8c500afe60 con 0x7f8c5006df10 2024-09-21T07:26:45.343 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 msgr2=0x7f8c3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 0x7f8c3007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8c501f8990 tx=0x7f8c4400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 msgr2=0x7f8c501f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8c4002b020 tx=0x7f8c40003f10 comp rx=0 tx=0).stop 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 shutdown_connections 2024-09-21T07:26:45.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8c30078bc0 0x7f8c3007b0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c500b02c0 0x7f8c500addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c50073ea0 0x7f8c500ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 --2- 172.21.15.105:0/2659951329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c5006df10 0x7f8c501f4c30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 >> 172.21.15.105:0/2659951329 conn(0x7f8c500d1270 msgr2=0x7f8c5015f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 shutdown_connections 2024-09-21T07:26:45.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45.344+0000 7f8c54920640 1 -- 172.21.15.105:0/2659951329 wait complete. 2024-09-21T07:26:45.356 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:26:45.357 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:26:45.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:45.357 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-09-21T07:26:45.372 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:45.372 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:45.373 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:26:45.374 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:45.374 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:26:45.374 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42769sec preferred_lft 42769sec 2024-09-21T07:26:45.374 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-21T07:26:45.374 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:45.374 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:26:45.374 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:45.374 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:45.374 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:26:45.374 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr flush dev ceph-brx 2024-09-21T07:26:45.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set ceph-brx up 2024-09-21T07:26:45.375 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:26:45.375 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:45.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:45.504 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:45.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:45.511 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:26:45.534 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-09-21T07:26:45.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:45.536 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-09-21T07:26:45.592 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-09-21T07:26:45.592 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:26:45.592 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:26:45.593 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:26:45.593 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:26:45.593 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:45.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:45.712 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:45.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:45.718 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:26:45.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:45.734 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-09-21T07:26:45.789 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:45.789 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:45.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:26:45.789 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T07:26:45.789 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:45.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:45.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:45.906 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T07:26:45.906 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:45.906 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T07:26:45.907 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:45.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:46.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.018 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:46.018 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:46.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-09-21T07:26:46.018 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:26:46.019 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:46.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:46.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:46.095 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:26:46.110 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:26:46.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:46.111 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:26:46.165 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:26:46.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:46.166 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:46.261 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:46.261 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:46.407 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:46.407 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@15ee7004-a3e2-41ce-b1eb-6358972a1ae2.cephfs=/ 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:26:46.408 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:26:46.410 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:26:46.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:46.410 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:26:46.434 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:46.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:46.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.531 INFO:teuthology.orchestra.run.smithi105.stdout:client4738 2024-09-21T07:26:46.532 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:46.532 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:46.550 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:46.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:46.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.644 INFO:teuthology.orchestra.run.smithi105.stdout:client4738 2024-09-21T07:26:46.645 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:46.692 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:46.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.699 INFO:teuthology.orchestra.run.smithi105.stdout:client4738 2024-09-21T07:26:46.700 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/15ee7004-a3e2-41ce-b1eb-6358972a1ae2.client4738/status') 2024-09-21T07:26:46.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:46.758 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4738 (3)172.21.15.105:0/1622103504 2024-09-21T07:26:46.758 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-09-21T07:26:46.759 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-09-21T07:26:46.759 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-09-21T07:26:46.759 INFO:teuthology.orchestra.run.smithi105.stderr:71 bytes copied, 2.198e-05 s, 3.2 MB/s 2024-09-21T07:26:46.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:46.765 INFO:tasks.kclient:client.2 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v2'} 2024-09-21T07:26:46.766 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:26:46.766 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:26:46.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:26:46.877 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 -- 172.21.15.105:0/1131603628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340ce870 msgr2=0x7f003415c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1131603628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340ce870 0x7f003415c730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f00340b3e20 tx=0x7f002402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 -- 172.21.15.105:0/1131603628 shutdown_connections 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1131603628 >> v2:172.21.15.112:3300/0 conn(0x7f00340b2ce0 0x7f00340b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1131603628 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340ce870 0x7f003415c730 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 -- 172.21.15.105:0/1131603628 >> 172.21.15.105:0/1131603628 conn(0x7f00340cdce0 msgr2=0x7f00340ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 -- 172.21.15.105:0/1131603628 shutdown_connections 2024-09-21T07:26:46.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.875+0000 7f003ae8b640 1 -- 172.21.15.105:0/1131603628 wait complete. 2024-09-21T07:26:46.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.877+0000 7f003ae8b640 1 Processor -- start 2024-09-21T07:26:46.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.877+0000 7f003ae8b640 1 -- start start 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00340ce870 0x7f0034148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 0x7f00340af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f00340b5d90 con 0x7f00340b2ce0 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00340b40b0 con 0x7f003415f2a0 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.878+0000 7f003ae8b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f003415fdc0 con 0x7f00340ce870 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60878/0 (socket says 172.21.15.105:60878) 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 -- 172.21.15.105:0/1284042099 learned_addr learned my addr 172.21.15.105:0/1284042099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:46.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0039428640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 0x7f00340af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0033fff640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00340ce870 0x7f0034148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 -- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00340ce870 msgr2=0x7f0034148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00340ce870 0x7f0034148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 -- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 msgr2=0x7f00340af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 0x7f00340af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 -- 172.21.15.105:0/1284042099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0034149420 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0039428640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 0x7f00340af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f0038c27640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0024009ca0 tx=0x7f00240039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024031990 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0024031c20 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240332d0 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00340ae1e0 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.879+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00340ae7a0 con 0x7f00340b2ce0 2024-09-21T07:26:46.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.880+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00340b30f0 con 0x7f00340b2ce0 2024-09-21T07:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.881+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0024033470 con 0x7f00340b2ce0 2024-09-21T07:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.881+0000 7f00317fa640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 0x7f002007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.881+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f00240bc6e0 con 0x7f00340b2ce0 2024-09-21T07:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.882+0000 7f0033fff640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 0x7f002007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:46.886 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.882+0000 7f0033fff640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 0x7f002007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00280058c0 tx=0x7f0028005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:46.887 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:46.886+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f002403f030 con 0x7f00340b2ce0 2024-09-21T07:26:47.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.099+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00340b5800 con 0x7f00340b2ce0 2024-09-21T07:26:47.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.100+0000 7f00317fa640 1 -- 172.21.15.105:0/1284042099 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f00240847e0 con 0x7f00340b2ce0 2024-09-21T07:26:47.102 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 msgr2=0x7f002007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 0x7f002007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f00280058c0 tx=0x7f0028005850 comp rx=0 tx=0).stop 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 msgr2=0x7f003406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0024009ca0 tx=0x7f00240039f0 comp rx=0 tx=0).stop 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 shutdown_connections 2024-09-21T07:26:47.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0020078b90 0x7f002007b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f003415f2a0 0x7f00340af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f00340ce870 0x7f0034148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 --2- 172.21.15.105:0/1284042099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00340b2ce0 0x7f003406d3d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 >> 172.21.15.105:0/1284042099 conn(0x7f00340cdce0 msgr2=0x7f003414d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 shutdown_connections 2024-09-21T07:26:47.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47.102+0000 7f003ae8b640 1 -- 172.21.15.105:0/1284042099 wait complete. 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:26:47.115 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:26:47.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.116 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-09-21T07:26:47.131 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-09-21T07:26:47.132 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42767sec preferred_lft 42767sec 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ee:f5:9f:7b:e4:c5 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.133 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::b417:e7ff:febd:7b35/64 scope link tentative 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run.smithi105.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ee:f5:9f:7b:e4:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ecf5:9fff:fe7b:e4c5/64 scope link tentative 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.134 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:26:47.195 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-09-21T07:26:47.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.196 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-09-21T07:26:47.252 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-09-21T07:26:47.252 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:26:47.252 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:26:47.252 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:26:47.252 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:26:47.253 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:47.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:47.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:47.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.379 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:26:47.394 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:26:47.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.395 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list-id 2024-09-21T07:26:47.450 INFO:teuthology.orchestra.run.smithi105.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T07:26:47.450 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:47.450 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:47.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:26:47.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T07:26:47.451 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:47.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:47.561 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:47.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.568 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:26:47.592 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:47.592 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether fa:35:1e:9c:26:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::f835:1eff:fe9c:268f/64 scope link 2024-09-21T07:26:47.593 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:47.659 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether fa:35:1e:9c:26:8f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:26:47.660 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:26:47.660 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.660 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::f835:1eff:fe9c:268f/64 scope link 2024-09-21T07:26:47.660 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:47.661 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T07:26:47.661 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:47.661 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:47.661 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T07:26:47.661 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:26:47.662 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T07:26:47.662 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T07:26:47.662 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T07:26:47.662 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:47.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:47.808 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:47.814 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:47.814 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:26:47.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.1 up 2024-09-21T07:26:47.815 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:26:47.815 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:26:47.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:47.885 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:47.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:47.892 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T07:26:47.906 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:26:47.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:47.907 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:26:47.962 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-21T07:26:47.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:47.962 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:48.056 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:48.056 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 2@15ee7004-a3e2-41ce-b1eb-6358972a1ae2.cephfs=/ 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.105:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:26:48.064 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:26:48.065 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:26:48.065 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:26:48.066 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:26:48.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:48.066 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:26:48.091 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:48.179 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:48.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:48.187 INFO:teuthology.orchestra.run.smithi105.stdout:client4866 2024-09-21T07:26:48.187 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:48.187 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:48.203 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:48.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:48.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:48.297 INFO:teuthology.orchestra.run.smithi105.stdout:client4866 2024-09-21T07:26:48.298 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:48.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:48.348 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:48.353 INFO:teuthology.orchestra.run.smithi105.stdout:client4866 2024-09-21T07:26:48.354 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/15ee7004-a3e2-41ce-b1eb-6358972a1ae2.client4866/status') 2024-09-21T07:26:48.402 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:48.412 INFO:teuthology.orchestra.run.smithi105.stdout:instance: client.4866 (3)192.168.0.2:0/972235166 2024-09-21T07:26:48.412 INFO:teuthology.orchestra.run.smithi105.stdout:blocklisted: false 2024-09-21T07:26:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records in 2024-09-21T07:26:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:0+1 records out 2024-09-21T07:26:48.412 INFO:teuthology.orchestra.run.smithi105.stderr:68 bytes copied, 2.1828e-05 s, 3.1 MB/s 2024-09-21T07:26:48.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:48.419 INFO:tasks.kclient:client.1 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v2'} 2024-09-21T07:26:48.419 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:26:48.419 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:26:48.420 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:26:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 -- 172.21.15.105:0/2890236534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880d0310 msgr2=0x7ff388167e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/2890236534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880d0310 0x7ff388167e00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff380005fd0 tx=0x7ff38002ab40 comp rx=0 tx=0).stop 2024-09-21T07:26:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 -- 172.21.15.105:0/2890236534 shutdown_connections 2024-09-21T07:26:48.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/2890236534 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880d0310 0x7ff388167e00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 -- 172.21.15.105:0/2890236534 >> 172.21.15.105:0/2890236534 conn(0x7ff3880cedb0 msgr2=0x7ff3880cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 -- 172.21.15.105:0/2890236534 shutdown_connections 2024-09-21T07:26:48.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.531+0000 7ff38e3c6640 1 -- 172.21.15.105:0/2890236534 wait complete. 2024-09-21T07:26:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.533+0000 7ff38e3c6640 1 Processor -- start 2024-09-21T07:26:48.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.533+0000 7ff38e3c6640 1 -- start start 2024-09-21T07:26:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 0x7ff38815fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3880d0310 0x7ff388160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880b7ed0 con 0x7ff3880b7940 2024-09-21T07:26:48.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff3880b7d50 con 0x7ff3880cff00 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38e3c6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff3880cea60 con 0x7ff3880d0310 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff387fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff387fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60898/0 (socket says 172.21.15.105:60898) 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff387fff640 1 -- 172.21.15.105:0/4159402861 learned_addr learned my addr 172.21.15.105:0/4159402861 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff3877fe640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 0x7ff38815fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.534+0000 7ff38c963640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3880d0310 0x7ff388160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 -- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3880d0310 msgr2=0x7ff388160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3880d0310 0x7ff388160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 -- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 msgr2=0x7ff38815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 0x7ff38815fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 -- 172.21.15.105:0/4159402861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3880c64e0 con 0x7ff3880b7940 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff3877fe640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 0x7ff38815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:48.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff387fff640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c370 tx=0x7ff37800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378017070 con 0x7ff3880b7940 2024-09-21T07:26:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3780035f0 con 0x7ff3880b7940 2024-09-21T07:26:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff388158e90 con 0x7ff3880b7940 2024-09-21T07:26:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378003a40 con 0x7ff3880b7940 2024-09-21T07:26:48.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.535+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3880c6780 con 0x7ff3880b7940 2024-09-21T07:26:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.536+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3880b17d0 con 0x7ff3880b7940 2024-09-21T07:26:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.537+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff37801a070 con 0x7ff3880b7940 2024-09-21T07:26:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.537+0000 7ff384ff9640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 0x7ff37407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.538+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff37809a320 con 0x7ff3880b7940 2024-09-21T07:26:48.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.538+0000 7ff3877fe640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 0x7ff37407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:48.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.538+0000 7ff3877fe640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 0x7ff37407b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007fd0 tx=0x7ff37c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:48.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.542+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff378062420 con 0x7ff3880b7940 2024-09-21T07:26:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.755+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3880b3c00 con 0x7ff3880b7940 2024-09-21T07:26:48.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.756+0000 7ff384ff9640 1 -- 172.21.15.105:0/4159402861 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff378061d20 con 0x7ff3880b7940 2024-09-21T07:26:48.758 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:26:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 msgr2=0x7ff37407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 0x7ff37407b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff37c007fd0 tx=0x7ff37c009040 comp rx=0 tx=0).stop 2024-09-21T07:26:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 msgr2=0x7ff388157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:48.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff37800c370 tx=0x7ff37800c830 comp rx=0 tx=0).stop 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 shutdown_connections 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff374078b90 0x7ff37407b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff3880cff00 0x7ff38815fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff3880d0310 0x7ff388160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 --2- 172.21.15.105:0/4159402861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3880b7940 0x7ff388157b20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:48.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 >> 172.21.15.105:0/4159402861 conn(0x7ff3880cedb0 msgr2=0x7ff3880724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:48.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 shutdown_connections 2024-09-21T07:26:48.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:48.758+0000 7ff38e3c6640 1 -- 172.21.15.105:0/4159402861 wait complete. 2024-09-21T07:26:48.771 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:26:48.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:26:48.773 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:26:48.773 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:26:48.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:48.773 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T07:26:48.792 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:48.792 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:48.792 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:48.792 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:48.792 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42745sec preferred_lft 42745sec 2024-09-21T07:26:48.793 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T07:26:48.794 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:48.794 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:26:48.794 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:48.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:48.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:48.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:48.997 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:26:49.021 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T07:26:49.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:49.023 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T07:26:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T07:26:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:26:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:26:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:26:49.112 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:26:49.112 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:49.113 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:49.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:26:49.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:26:49.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:26:49.113 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:49.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:49.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:49.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:49.365 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:26:49.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:49.380 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T07:26:49.436 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:49.436 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:49.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:26:49.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T07:26:49.437 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:49.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:49.553 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T07:26:49.553 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T07:26:49.554 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:49.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:49.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:49.679 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:49.679 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:49.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-09-21T07:26:49.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:26:49.680 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:49.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:49.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:49.756 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:49.756 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:26:49.772 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:26:49.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:49.772 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:26:49.827 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-21T07:26:49.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:49.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:49.927 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:49.927 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-09-21T07:26:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 1@15ee7004-a3e2-41ce-b1eb-6358972a1ae2.cephfs=/ 2024-09-21T07:26:50.044 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.105:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-09-21T07:26:50.045 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:26:50.045 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:26:50.045 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:26:50.045 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:26:50.046 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:26:50.047 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:26:50.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:50.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:26:50.076 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:50.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:50.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:50.172 INFO:teuthology.orchestra.run.smithi112.stdout:client4589 2024-09-21T07:26:50.173 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:50.173 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:50.189 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:50.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:50.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:50.283 INFO:teuthology.orchestra.run.smithi112.stdout:client4589 2024-09-21T07:26:50.284 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:50.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:50.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:50.338 INFO:teuthology.orchestra.run.smithi112.stdout:client4589 2024-09-21T07:26:50.338 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/15ee7004-a3e2-41ce-b1eb-6358972a1ae2.client4589/status') 2024-09-21T07:26:50.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:50.396 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4589 (3)192.168.0.1:0/2505638842 2024-09-21T07:26:50.396 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-09-21T07:26:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-09-21T07:26:50.396 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-09-21T07:26:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:69 bytes copied, 3.9785e-05 s, 1.7 MB/s 2024-09-21T07:26:50.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:50.403 INFO:tasks.kclient:client.3 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v2'} 2024-09-21T07:26:50.403 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:26:50.403 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:26:50.403 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:26:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 -- 172.21.15.105:0/3257012139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0ceb50 msgr2=0x7ff60c0cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 --2- 172.21.15.105:0/3257012139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0cef60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff60c06e600 tx=0x7ff60402aa90 comp rx=0 tx=0).stop 2024-09-21T07:26:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 -- 172.21.15.105:0/3257012139 shutdown_connections 2024-09-21T07:26:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 --2- 172.21.15.105:0/3257012139 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 0x7ff60c06f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.530 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 --2- 172.21.15.105:0/3257012139 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0cef60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 -- 172.21.15.105:0/3257012139 >> 172.21.15.105:0/3257012139 conn(0x7ff60c09d850 msgr2=0x7ff60c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:50.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 -- 172.21.15.105:0/3257012139 shutdown_connections 2024-09-21T07:26:50.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.528+0000 7ff61219b640 1 -- 172.21.15.105:0/3257012139 wait complete. 2024-09-21T07:26:50.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.530+0000 7ff61219b640 1 Processor -- start 2024-09-21T07:26:50.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.530+0000 7ff61219b640 1 -- start start 2024-09-21T07:26:50.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:50.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 0x7ff60c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:50.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0d0e70 con 0x7ff60c0b3710 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff60c0d0cf0 con 0x7ff60c0ceb50 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff61219b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff60c0d0ff0 con 0x7ff60c0b9790 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff60b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38776/0 (socket says 172.21.15.105:38776) 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff60b7fe640 1 -- 172.21.15.105:0/795794748 learned_addr learned my addr 172.21.15.105:0/795794748 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff60affd640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 0x7ff60c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.531+0000 7ff60bfff640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 -- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 msgr2=0x7ff60c0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 0x7ff60c0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 -- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff60c0ceb50 msgr2=0x7ff60c0b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 -- 172.21.15.105:0/795794748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff60c0cc4c0 con 0x7ff60c0b3710 2024-09-21T07:26:50.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff60b7fe640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c3c0 tx=0x7ff5f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:50.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4017070 con 0x7ff60c0b3710 2024-09-21T07:26:50.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5f40053c0 con 0x7ff60c0b3710 2024-09-21T07:26:50.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff60c1f8e30 con 0x7ff60c0b3710 2024-09-21T07:26:50.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4003880 con 0x7ff60c0b3710 2024-09-21T07:26:50.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.532+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff60c1f93f0 con 0x7ff60c0b3710 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.533+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60c0d1530 con 0x7ff60c0b3710 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.534+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5f401a070 con 0x7ff60c0b3710 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.534+0000 7ff603fff640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 0x7ff5ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.534+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff5f409a230 con 0x7ff60c0b3710 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.535+0000 7ff60affd640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 0x7ff5ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:50.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.535+0000 7ff60affd640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 0x7ff5ec07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff60c15cc00 tx=0x7ff5fc005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:50.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.539+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5f4061d70 con 0x7ff60c0b3710 2024-09-21T07:26:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.752+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff60c068e30 con 0x7ff60c0b3710 2024-09-21T07:26:50.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.752+0000 7ff603fff640 1 -- 172.21.15.105:0/795794748 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff5f40667a0 con 0x7ff60c0b3710 2024-09-21T07:26:50.754 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.754+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 msgr2=0x7ff5ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.754+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 0x7ff5ec07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff60c15cc00 tx=0x7ff5fc005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.754+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 msgr2=0x7ff60c15c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400c3c0 tx=0x7ff5f400c880 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 shutdown_connections 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff5ec078bc0 0x7ff5ec07b0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff60c0ceb50 0x7ff60c0b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff60c0b9790 0x7ff60c0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 --2- 172.21.15.105:0/795794748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff60c0b3710 0x7ff60c15c0f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 >> 172.21.15.105:0/795794748 conn(0x7ff60c09d850 msgr2=0x7ff60c1619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 shutdown_connections 2024-09-21T07:26:50.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:50.755+0000 7ff61219b640 1 -- 172.21.15.105:0/795794748 wait complete. 2024-09-21T07:26:50.770 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:26:50.771 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:26:50.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:50.771 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-09-21T07:26:50.788 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42743sec preferred_lft 42743sec 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link noprefixroute 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 92:a8:05:2f:cf:d6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.789 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::cc30:e8ff:febf:98aa/64 scope link 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 92:a8:05:2f:cf:d6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::90a8:5ff:fe2f:cfd6/64 scope link tentative 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:50.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:50.790 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:26:50.852 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-09-21T07:26:50.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:50.854 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-09-21T07:26:50.910 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-09-21T07:26:50.911 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:26:50.911 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:26:50.911 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:26:50.911 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:26:50.911 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:50.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:51.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.036 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:26:51.051 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:26:51.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.052 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-09-21T07:26:51.106 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T07:26:51.107 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:51.107 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:51.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:26:51.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T07:26:51.107 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:51.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:51.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 96:3a:5e:ce:9a:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.247 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::943a:5eff:fece:9ab2/64 scope link 2024-09-21T07:26:51.248 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:26:51.313 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:26:51.313 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:26:51.313 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 96:3a:5e:ce:9a:b2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::943a:5eff:fece:9ab2/64 scope link 2024-09-21T07:26:51.314 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:26:51.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T07:26:51.315 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:51.315 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:51.315 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T07:26:51.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:26:51.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T07:26:51.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T07:26:51.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T07:26:51.316 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:51.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:51.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.467 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:26:51.467 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-09-21T07:26:51.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2024-09-21T07:26:51.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:26:51.468 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-09-21T07:26:51.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:51.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:51.543 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T07:26:51.558 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:26:51.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:26:51.559 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:26:51.617 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-21T07:26:51.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:51.711 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-09-21T07:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 3@15ee7004-a3e2-41ce-b1eb-6358972a1ae2.cephfs=/ 2024-09-21T07:26:51.720 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.105:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-09-21T07:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T07:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:26:51.721 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:26:51.722 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:26:51.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:51.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:26:51.746 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:51.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:51.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.843 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T07:26:51.843 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:51.843 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:51.859 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:51.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:51.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:51.955 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T07:26:51.955 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:26:52.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:26:52.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:52.010 INFO:teuthology.orchestra.run.smithi112.stdout:client4598 2024-09-21T07:26:52.011 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/15ee7004-a3e2-41ce-b1eb-6358972a1ae2.client4598/status') 2024-09-21T07:26:52.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:26:52.072 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.4598 (3)192.168.0.2:0/3956189590 2024-09-21T07:26:52.072 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-09-21T07:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-09-21T07:26:52.072 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-09-21T07:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:69 bytes copied, 2.4597e-05 s, 2.8 MB/s 2024-09-21T07:26:52.073 INFO:teuthology.orchestra.run.smithi112.stderr:2024-09-21T07:26:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:26:52.079 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T07:26:52.145 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2024-09-21T07:26:52.145 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2024-09-21T07:26:52.145 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:52.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 -- 172.21.15.105:0/2492102561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80d0310 msgr2=0x7f68a8167e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 --2- 172.21.15.105:0/2492102561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80d0310 0x7f68a8167e00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005fd0 tx=0x7f68a002a940 comp rx=0 tx=0).stop 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 -- 172.21.15.105:0/2492102561 shutdown_connections 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 --2- 172.21.15.105:0/2492102561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80d0310 0x7f68a8167e00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 -- 172.21.15.105:0/2492102561 >> 172.21.15.105:0/2492102561 conn(0x7f68a80cedb0 msgr2=0x7f68a80cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 -- 172.21.15.105:0/2492102561 shutdown_connections 2024-09-21T07:26:52.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.271+0000 7f68ac80e640 1 -- 172.21.15.105:0/2492102561 wait complete. 2024-09-21T07:26:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.273+0000 7f68ac80e640 1 Processor -- start 2024-09-21T07:26:52.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.273+0000 7f68ac80e640 1 -- start start 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a80cff00 0x7f68a815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68a80d0310 0x7f68a8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80b7ed0 con 0x7f68a80b7940 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f68a80b7d50 con 0x7f68a80cff00 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68ac80e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f68a80cea60 con 0x7f68a80d0310 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38794/0 (socket says 172.21.15.105:38794) 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 -- 172.21.15.105:0/458257151 learned_addr learned my addr 172.21.15.105:0/458257151 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a6d9d640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68a80d0310 0x7f68a8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f689ffff640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a80cff00 0x7f68a815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 -- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68a80d0310 msgr2=0x7f68a8153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68a80d0310 0x7f68a8153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 -- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a80cff00 msgr2=0x7f68a815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a80cff00 0x7f68a815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 -- 172.21.15.105:0/458257151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a815e500 con 0x7f68a80b7940 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.275+0000 7f68a659c640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f689800c3a0 tx=0x7f689800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.276+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898017070 con 0x7f68a80b7940 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.276+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68980035f0 con 0x7f68a80b7940 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.276+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898003a40 con 0x7f68a80b7940 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.276+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a815df40 con 0x7f68a80b7940 2024-09-21T07:26:52.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.276+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a80c5e90 con 0x7f68a80b7940 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.277+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a80b17d0 con 0x7f68a80b7940 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.277+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f689801a070 con 0x7f68a80b7940 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.278+0000 7f689effd640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 0x7f688807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.278+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f689809a270 con 0x7f68a80b7940 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.278+0000 7f689ffff640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 0x7f688807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.278+0000 7f689ffff640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 0x7f688807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68a815edb0 tx=0x7f6890009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.284 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.282+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6898061d20 con 0x7f68a80b7940 2024-09-21T07:26:52.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.459+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68a80b3c00 con 0x7f68a80b7940 2024-09-21T07:26:52.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.460+0000 7f689effd640 1 -- 172.21.15.105:0/458257151 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18670 (secure 0 0 0) 0x7f6898066750 con 0x7f68a80b7940 2024-09-21T07:26:52.463 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:52.463 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":27,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:21.785810+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 msgr2=0x7f688807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 0x7f688807b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68a815edb0 tx=0x7f6890009040 comp rx=0 tx=0).stop 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 msgr2=0x7f68a815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f689800c3a0 tx=0x7f689800c860 comp rx=0 tx=0).stop 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 shutdown_connections 2024-09-21T07:26:52.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6888078bc0 0x7f688807b0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68a80cff00 0x7f68a815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f68a80d0310 0x7f68a8153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 --2- 172.21.15.105:0/458257151 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68a80b7940 0x7f68a815d5d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 >> 172.21.15.105:0/458257151 conn(0x7f68a80cedb0 msgr2=0x7f68a8072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 shutdown_connections 2024-09-21T07:26:52.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.463+0000 7f68ac80e640 1 -- 172.21.15.105:0/458257151 wait complete. 2024-09-21T07:26:52.477 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:26:04.782684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T07:26:13.030140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T07:26:14.056565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-21T07:26:17.764268+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-21T07:26:52.477 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 -- 172.21.15.105:0/3392240300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee580d01e0 msgr2=0x7fee580b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3392240300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee580d01e0 0x7fee580b1a50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fee44009830 tx=0x7fee4402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 -- 172.21.15.105:0/3392240300 shutdown_connections 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3392240300 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee580d01e0 0x7fee580b1a50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 -- 172.21.15.105:0/3392240300 >> 172.21.15.105:0/3392240300 conn(0x7fee580cedf0 msgr2=0x7fee580cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 -- 172.21.15.105:0/3392240300 shutdown_connections 2024-09-21T07:26:52.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.594+0000 7fee5ead5640 1 -- 172.21.15.105:0/3392240300 wait complete. 2024-09-21T07:26:52.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.595+0000 7fee5ead5640 1 Processor -- start 2024-09-21T07:26:52.597 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.595+0000 7fee5ead5640 1 -- start start 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 0x7fee5814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 0x7fee580c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fee58167e50 con 0x7fee580d01e0 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee58167cd0 con 0x7fee5809ba30 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5ead5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fee58167fd0 con 0x7fee580d05f0 2024-09-21T07:26:52.600 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38796/0 (socket says 172.21.15.105:38796) 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 -- 172.21.15.105:0/3816534381 learned_addr learned my addr 172.21.15.105:0/3816534381 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5c871640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 0x7fee5814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee5d072640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 0x7fee580c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 -- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 msgr2=0x7fee580c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 0x7fee580c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 -- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 msgr2=0x7fee5814bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.597+0000 7fee57fff640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 0x7fee5814bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee57fff640 1 -- 172.21.15.105:0/3816534381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580cc2a0 con 0x7fee580d01e0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee5d072640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 0x7fee580c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee5c871640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 0x7fee5814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee57fff640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee4402abc0 tx=0x7fee44030650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4403f070 con 0x7fee580d01e0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee44030d40 con 0x7fee580d01e0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee580cb500 con 0x7fee580d01e0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44004280 con 0x7fee580d01e0 2024-09-21T07:26:52.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.598+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee580cbce0 con 0x7fee580d01e0 2024-09-21T07:26:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.599+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee580ce550 con 0x7fee580d01e0 2024-09-21T07:26:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.600+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee440035f0 con 0x7fee580d01e0 2024-09-21T07:26:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.601+0000 7fee557fa640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 0x7fee3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.603 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.601+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fee440bb980 con 0x7fee580d01e0 2024-09-21T07:26:52.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.601+0000 7fee5c871640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 0x7fee3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.604 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.602+0000 7fee5c871640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 0x7fee3c07b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000fb0 tx=0x7fee4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.606+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee440834c0 con 0x7fee580d01e0 2024-09-21T07:26:52.793 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.791+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee58155fc0 con 0x7fee580d01e0 2024-09-21T07:26:52.794 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.792+0000 7fee557fa640 1 -- 172.21.15.105:0/3816534381 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fee44087ef0 con 0x7fee580d01e0 2024-09-21T07:26:52.794 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 msgr2=0x7fee3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 0x7fee3c07b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000fb0 tx=0x7fee4c009040 comp rx=0 tx=0).stop 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 msgr2=0x7fee5814c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fee4402abc0 tx=0x7fee44030650 comp rx=0 tx=0).stop 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 shutdown_connections 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fee3c078bc0 0x7fee3c07b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fee580d05f0 0x7fee580c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.796 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee580d01e0 0x7fee5814c210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 --2- 172.21.15.105:0/3816534381 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee5809ba30 0x7fee5814bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 >> 172.21.15.105:0/3816534381 conn(0x7fee580cedf0 msgr2=0x7fee580b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 shutdown_connections 2024-09-21T07:26:52.797 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.794+0000 7fee5ead5640 1 -- 172.21.15.105:0/3816534381 wait complete. 2024-09-21T07:26:52.807 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:26:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1893696052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 msgr2=0x7fa174074830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1893696052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa174074830 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa16c005fd0 tx=0x7fa16c02a8d0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1893696052 shutdown_connections 2024-09-21T07:26:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1893696052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa174074830 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1893696052 >> 172.21.15.105:0/1893696052 conn(0x7fa1740cdce0 msgr2=0x7fa1740ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:52.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1893696052 shutdown_connections 2024-09-21T07:26:52.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.925+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1893696052 wait complete. 2024-09-21T07:26:52.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.927+0000 7fa17a4b4640 1 Processor -- start 2024-09-21T07:26:52.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.927+0000 7fa17a4b4640 1 -- start start 2024-09-21T07:26:52.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.928+0000 7fa17a4b4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 0x7fa1740b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa17a4b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa1740c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa17a4b4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa17a4b4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa174074d70 con 0x7fa1740b3340 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa17a4b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740cea10 con 0x7fa17415c670 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa17a4b4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa174074ef0 con 0x7fa1740701e0 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa178a51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa178a51640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38802/0 (socket says 172.21.15.105:38802) 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa178a51640 1 -- 172.21.15.105:0/1573881729 learned_addr learned my addr 172.21.15.105:0/1573881729 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 0x7fa1740b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa1737fe640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa1740c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 -- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 msgr2=0x7fa1740c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa1740c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 -- 172.21.15.105:0/1573881729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 msgr2=0x7fa1740c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:52.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 -- 172.21.15.105:0/1573881729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740c4ec0 con 0x7fa1740701e0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa178a51640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.929+0000 7fa173fff640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 0x7fa1740b2260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c500 tx=0x7fa16400c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.930+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164016020 con 0x7fa1740701e0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.930+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa164012940 con 0x7fa1740701e0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.930+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1740c51a0 con 0x7fa1740701e0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.930+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164012d90 con 0x7fa1740701e0 2024-09-21T07:26:52.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.930+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1740c5760 con 0x7fa1740701e0 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.931+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17406cda0 con 0x7fa1740701e0 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.931+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa164019020 con 0x7fa1740701e0 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.932+0000 7fa170ff9640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 0x7fa16007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.932+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa16409abb0 con 0x7fa1740701e0 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.932+0000 7fa1737fe640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 0x7fa16007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:52.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.933+0000 7fa1737fe640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 0x7fa16007b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa168000f60 tx=0x7fa168009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:52.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:52.937+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa164020030 con 0x7fa1740701e0 2024-09-21T07:26:53.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.125+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1740ce470 con 0x7fa1740701e0 2024-09-21T07:26:53.128 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.126+0000 7fa170ff9640 1 -- 172.21.15.105:0/1573881729 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa164062700 con 0x7fa1740701e0 2024-09-21T07:26:53.128 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-09-21T07:26:53.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 msgr2=0x7fa16007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 0x7fa16007b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa168000f60 tx=0x7fa168009040 comp rx=0 tx=0).stop 2024-09-21T07:26:53.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 msgr2=0x7fa1740b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 0x7fa1740b2260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa16400c500 tx=0x7fa16400c9c0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 shutdown_connections 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa160078b90 0x7fa16007b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa17415c670 0x7fa1740c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa1740b3340 0x7fa1740c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 --2- 172.21.15.105:0/1573881729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1740701e0 0x7fa1740b2260 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 >> 172.21.15.105:0/1573881729 conn(0x7fa1740cdce0 msgr2=0x7fa1740d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 shutdown_connections 2024-09-21T07:26:53.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.128+0000 7fa17a4b4640 1 -- 172.21.15.105:0/1573881729 wait complete. 2024-09-21T07:26:53.143 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:26:53.261 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 -- 172.21.15.105:0/384515428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8160270 msgr2=0x7f9ae8160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 --2- 172.21.15.105:0/384515428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8160270 0x7f9ae8160680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ae809dea0 tx=0x7f9ad002a560 comp rx=0 tx=0).stop 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 -- 172.21.15.105:0/384515428 shutdown_connections 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 --2- 172.21.15.105:0/384515428 >> v2:172.21.15.112:3300/0 conn(0x7f9ae815bd80 0x7f9ae815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 --2- 172.21.15.105:0/384515428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae8160270 0x7f9ae8160680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 --2- 172.21.15.105:0/384515428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae80af450 0x7f9ae8073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 -- 172.21.15.105:0/384515428 >> 172.21.15.105:0/384515428 conn(0x7f9ae80ccb90 msgr2=0x7f9ae80ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 -- 172.21.15.105:0/384515428 shutdown_connections 2024-09-21T07:26:53.262 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.259+0000 7f9aedd28640 1 -- 172.21.15.105:0/384515428 wait complete. 2024-09-21T07:26:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.261+0000 7f9aedd28640 1 Processor -- start 2024-09-21T07:26:53.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.261+0000 7f9aedd28640 1 -- start start 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 0x7f9ae80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 0x7f9ae815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae815c850 con 0x7f9ae80af450 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae815c6d0 con 0x7f9ae815bd80 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae815c9d0 con 0x7f9ae8160270 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38814/0 (socket says 172.21.15.105:38814) 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 -- 172.21.15.105:0/3331268034 learned_addr learned my addr 172.21.15.105:0/3331268034 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9adffff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 0x7f9ae80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae7fff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 0x7f9ae815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.266 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 -- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 msgr2=0x7f9ae815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 0x7f9ae815d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 -- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 msgr2=0x7f9ae80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 0x7f9ae80b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 -- 172.21.15.105:0/3331268034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae815f9a0 con 0x7f9ae80af450 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9adffff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 0x7f9ae80b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae7fff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 0x7f9ae815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae77fe640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e7c0 tx=0x7f9ad800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800cc60 con 0x7f9ae80af450 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad800ce00 con 0x7f9ae80af450 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae80c97b0 con 0x7f9ae80af450 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8005280 con 0x7f9ae80af450 2024-09-21T07:26:53.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.263+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae815ce90 con 0x7f9ae80af450 2024-09-21T07:26:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.265+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae815c190 con 0x7f9ae80af450 2024-09-21T07:26:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.265+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ad8005420 con 0x7f9ae80af450 2024-09-21T07:26:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.266+0000 7f9ae4ff9640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 0x7f9ac807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.266+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9ad809b870 con 0x7f9ae80af450 2024-09-21T07:26:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.266+0000 7f9adffff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 0x7f9ac807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.266+0000 7f9adffff640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 0x7f9ac807b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0033000 tx=0x7f9ad0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.272 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.270+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ae815f9a0 con 0x7f9ae80af450 2024-09-21T07:26:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.451+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ae806fd00 con 0x7f9ae80af450 2024-09-21T07:26:53.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.451+0000 7f9ae4ff9640 1 -- 172.21.15.105:0/3331268034 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f9ad8063a00 con 0x7f9ae80af450 2024-09-21T07:26:53.453 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 64 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 msgr2=0x7f9ac807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 0x7f9ac807b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0033000 tx=0x7f9ad0004730 comp rx=0 tx=0).stop 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 msgr2=0x7f9ae815b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800e7c0 tx=0x7f9ad800ec80 comp rx=0 tx=0).stop 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 shutdown_connections 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ac8078b90 0x7f9ac807b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ae815bd80 0x7f9ae80b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9ae8160270 0x7f9ae815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 --2- 172.21.15.105:0/3331268034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9ae80af450 0x7f9ae815b4a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 >> 172.21.15.105:0/3331268034 conn(0x7f9ae80ccb90 msgr2=0x7f9ae806d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 shutdown_connections 2024-09-21T07:26:53.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.454+0000 7f9aedd28640 1 -- 172.21.15.105:0/3331268034 wait complete. 2024-09-21T07:26:53.467 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.581+0000 7fb2f6815640 1 -- 172.21.15.105:0/3887796237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b0ce0 msgr2=0x7fb2f00b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.581+0000 7fb2f6815640 1 --2- 172.21.15.105:0/3887796237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b0ce0 0x7fb2f00b4860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005fd0 tx=0x7fb2e802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.581+0000 7fb2f6815640 1 -- 172.21.15.105:0/3887796237 shutdown_connections 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.581+0000 7fb2f6815640 1 --2- 172.21.15.105:0/3887796237 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b0ce0 0x7fb2f00b4860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.581+0000 7fb2f6815640 1 -- 172.21.15.105:0/3887796237 >> 172.21.15.105:0/3887796237 conn(0x7fb2f00b6670 msgr2=0x7fb2f00b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.582+0000 7fb2f6815640 1 -- 172.21.15.105:0/3887796237 shutdown_connections 2024-09-21T07:26:53.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.582+0000 7fb2f6815640 1 -- 172.21.15.105:0/3887796237 wait complete. 2024-09-21T07:26:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.583+0000 7fb2f6815640 1 Processor -- start 2024-09-21T07:26:53.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.583+0000 7fb2f6815640 1 -- start start 2024-09-21T07:26:53.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2f00b0ce0 0x7fb2f00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 0x7fb2f0153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f015bae0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f00b6de0 con 0x7fb2f015b6d0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f6815640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f015bc60 con 0x7fb2f00b0ce0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38822/0 (socket says 172.21.15.105:38822) 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 -- 172.21.15.105:0/1569745134 learned_addr learned my addr 172.21.15.105:0/1569745134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2ef7fe640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2f00b0ce0 0x7fb2f00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f4db2640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 0x7fb2f0153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 -- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2f00b0ce0 msgr2=0x7fb2f00cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2f00b0ce0 0x7fb2f00cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 -- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 msgr2=0x7fb2f0153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 0x7fb2f0153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 -- 172.21.15.105:0/1569745134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f00d0520 con 0x7fb2f00b08d0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2f4db2640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 0x7fb2f0153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2effff640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c400 tx=0x7fb2e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.585+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0017070 con 0x7fb2f00b08d0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.586+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2e00053c0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.586+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f00d0800 con 0x7fb2f00b08d0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.586+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0003850 con 0x7fb2f00b08d0 2024-09-21T07:26:53.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.586+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f00d0dc0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.587+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f00b4fd0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.587+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2e001a070 con 0x7fb2f00b08d0 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.588+0000 7fb2ecff9640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.588+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb2e009a340 con 0x7fb2f00b08d0 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.588+0000 7fb2ef7fe640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.590 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.588+0000 7fb2ef7fe640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb2f009f690 tx=0x7fb2d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.594 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.592+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2e0061e80 con 0x7fb2f00b08d0 2024-09-21T07:26:53.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.775+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb2f01550e0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.775+0000 7fb2ecff9640 1 -- 172.21.15.105:0/1569745134 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb2e00668b0 con 0x7fb2f00b08d0 2024-09-21T07:26:53.777 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 msgr2=0x7fb2d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 0x7fb2d007b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb2f009f690 tx=0x7fb2d8009040 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 msgr2=0x7fb2f0143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000c400 tx=0x7fb2e000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 shutdown_connections 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb2d0078b90 0x7fb2d007b0a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb2f015b6d0 0x7fb2f0153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb2f00b0ce0 0x7fb2f00cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 --2- 172.21.15.105:0/1569745134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb2f00b08d0 0x7fb2f0143c60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 >> 172.21.15.105:0/1569745134 conn(0x7fb2f00b6670 msgr2=0x7fb2f00718f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 shutdown_connections 2024-09-21T07:26:53.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.778+0000 7fb2f6815640 1 -- 172.21.15.105:0/1569745134 wait complete. 2024-09-21T07:26:53.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-09-21T07:26:53.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 -- 172.21.15.105:0/3229403066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14000b3800 msgr2=0x7f14000b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 --2- 172.21.15.105:0/3229403066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14000b3800 0x7f14000b5fc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f14000a2390 tx=0x7f13fc02a530 comp rx=0 tx=0).stop 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 -- 172.21.15.105:0/3229403066 shutdown_connections 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 --2- 172.21.15.105:0/3229403066 >> v2:172.21.15.112:3300/0 conn(0x7f140006e010 0x7f14000b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 --2- 172.21.15.105:0/3229403066 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f14000b3800 0x7f14000b5fc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 -- 172.21.15.105:0/3229403066 >> 172.21.15.105:0/3229403066 conn(0x7f140009d810 msgr2=0x7f140008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 -- 172.21.15.105:0/3229403066 shutdown_connections 2024-09-21T07:26:53.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.900+0000 7f1406cec640 1 -- 172.21.15.105:0/3229403066 wait complete. 2024-09-21T07:26:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.902+0000 7f1406cec640 1 Processor -- start 2024-09-21T07:26:53.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.902+0000 7f1406cec640 1 -- start start 2024-09-21T07:26:53.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14000b3800 0x7f140015b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14000d16f0 0x7f14000cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f14000b2350 con 0x7f140006e010 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14000b21d0 con 0x7f14000b3800 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1406cec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f14000b24d0 con 0x7f14000d16f0 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38842/0 (socket says 172.21.15.105:38842) 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 -- 172.21.15.105:0/1345409110 learned_addr learned my addr 172.21.15.105:0/1345409110 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f13f7fff640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14000b3800 0x7f140015b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1405289640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14000d16f0 0x7f14000cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 -- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14000d16f0 msgr2=0x7f14000cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14000d16f0 0x7f14000cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 -- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14000b3800 msgr2=0x7f140015b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14000b3800 0x7f140015b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 -- 172.21.15.105:0/1345409110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14001f5440 con 0x7f140006e010 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f1404a88640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009910 tx=0x7f13fc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.904+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc03f070 con 0x7f140006e010 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.905+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13fc004340 con 0x7f140006e010 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.905+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14001f5720 con 0x7f140006e010 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.905+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc031bb0 con 0x7f140006e010 2024-09-21T07:26:53.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.905+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14001f5ce0 con 0x7f140006e010 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.905+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14000b29f0 con 0x7f140006e010 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.906+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13fc042070 con 0x7f140006e010 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.907+0000 7f13f57fa640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 0x7f13ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.907+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f13fc0bc020 con 0x7f140006e010 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.907+0000 7f13f7fff640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 0x7f13ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:53.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.907+0000 7f13f7fff640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 0x7f13ec07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14001619b0 tx=0x7f13f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:53.913 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:53.911+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13fc031770 con 0x7f140006e010 2024-09-21T07:26:54.158 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.156+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f14000691d0 con 0x7f140006e010 2024-09-21T07:26:54.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.920+0000 7f13f57fa640 1 -- 172.21.15.105:0/1345409110 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v55) ==== 158+0+0 (secure 0 0 0) 0x7f13fc083a50 con 0x7f140006e010 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 msgr2=0x7f13ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 0x7f13ec07b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14001619b0 tx=0x7f13f0009040 comp rx=0 tx=0).stop 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 msgr2=0x7f1400158d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13fc009910 tx=0x7f13fc005270 comp rx=0 tx=0).stop 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 shutdown_connections 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f13ec078b90 0x7f13ec07b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14000b3800 0x7f140015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14000d16f0 0x7f14000cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 --2- 172.21.15.105:0/1345409110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f140006e010 0x7f1400158d30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:54.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 >> 172.21.15.105:0/1345409110 conn(0x7f140009d810 msgr2=0x7f14000b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:54.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 shutdown_connections 2024-09-21T07:26:54.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:54.923+0000 7f1406cec640 1 -- 172.21.15.105:0/1345409110 wait complete. 2024-09-21T07:26:54.940 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 -- 172.21.15.105:0/707628594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 msgr2=0x7f7b980ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/707628594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980ca760 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7b980cc0d0 tx=0x7f7b9402a410 comp rx=0 tx=0).stop 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 -- 172.21.15.105:0/707628594 shutdown_connections 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/707628594 >> v2:172.21.15.112:3300/0 conn(0x7f7b980cb5f0 0x7f7b9806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/707628594 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980ca760 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 -- 172.21.15.105:0/707628594 >> 172.21.15.105:0/707628594 conn(0x7f7b980ccb90 msgr2=0x7f7b980ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:55.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 -- 172.21.15.105:0/707628594 shutdown_connections 2024-09-21T07:26:55.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.049+0000 7f7b9eab4640 1 -- 172.21.15.105:0/707628594 wait complete. 2024-09-21T07:26:55.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.051+0000 7f7b9eab4640 1 Processor -- start 2024-09-21T07:26:55.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.051+0000 7f7b9eab4640 1 -- start start 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b980cb5f0 0x7f7b98158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b980cba00 0x7f7b980b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9815f030 con 0x7f7b98074d00 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7b980700a0 con 0x7f7b980cb5f0 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9eab4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9815f1b0 con 0x7f7b980cba00 2024-09-21T07:26:55.055 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9c850640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9c850640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38872/0 (socket says 172.21.15.105:38872) 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.052+0000 7f7b9c850640 1 -- 172.21.15.105:0/2165056593 learned_addr learned my addr 172.21.15.105:0/2165056593 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 -- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b980cba00 msgr2=0x7f7b980b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b980cba00 0x7f7b980b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 -- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b980cb5f0 msgr2=0x7f7b98158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b8ffff640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b980cb5f0 0x7f7b98158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b980cb5f0 0x7f7b98158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 -- 172.21.15.105:0/2165056593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b981f25c0 con 0x7f7b98074d00 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9c850640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402aa00 tx=0x7f7b94033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94005c90 con 0x7f7b98074d00 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b94031e50 con 0x7f7b98074d00 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b98158bc0 con 0x7f7b98074d00 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94005260 con 0x7f7b98074d00 2024-09-21T07:26:55.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.053+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b980b4470 con 0x7f7b98074d00 2024-09-21T07:26:55.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.054+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b981f2790 con 0x7f7b98074d00 2024-09-21T07:26:55.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.055+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b94005400 con 0x7f7b98074d00 2024-09-21T07:26:55.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.055+0000 7f7b8d7fa640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 0x7f7b8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.055+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7f7b940bc2d0 con 0x7f7b98074d00 2024-09-21T07:26:55.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.056+0000 7f7b8ffff640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 0x7f7b8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.058 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.056+0000 7f7b8ffff640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 0x7f7b8407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b9809e280 tx=0x7f7b88005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.060+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7b9403f030 con 0x7f7b98074d00 2024-09-21T07:26:55.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.239+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7b980ca760 con 0x7f7b98074d00 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.240+0000 7f7b8d7fa640 1 -- 172.21.15.105:0/2165056593 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7f7b94031770 con 0x7f7b98074d00 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_client 20/20 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:26:55.242 INFO:teuthology.orchestra.run.smithi105.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4841.529353 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4923.673897 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4851.413707 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 8482.139680 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4351.294472 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 5941.555027 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4066.583026 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3965.579044 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_mds 20/20 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced debug_ms 2/2 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_debug_frag true 2024-09-21T07:26:55.243 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:mds dev mds_verify_scatter true 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced client_mount_timeout 600 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced debug_client 20/20 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced debug_ms 1/1 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:26:55.244 INFO:teuthology.orchestra.run.smithi105.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:26:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 msgr2=0x7f7b8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 0x7f7b8407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b9809e280 tx=0x7f7b88005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 msgr2=0x7f7b980b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402aa00 tx=0x7f7b94033c20 comp rx=0 tx=0).stop 2024-09-21T07:26:55.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 shutdown_connections 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7b84078b90 0x7f7b8407b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b980cb5f0 0x7f7b98158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b980cba00 0x7f7b980b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 --2- 172.21.15.105:0/2165056593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7b98074d00 0x7f7b980b59a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 >> 172.21.15.105:0/2165056593 conn(0x7f7b980ccb90 msgr2=0x7f7b980aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 shutdown_connections 2024-09-21T07:26:55.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.242+0000 7f7b9eab4640 1 -- 172.21.15.105:0/2165056593 wait complete. 2024-09-21T07:26:55.255 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:26:55.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 -- 172.21.15.105:0/994108726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d0200 msgr2=0x7fb858152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/994108726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d0200 0x7fb858152630 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb840009830 tx=0x7fb84002a3f0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 -- 172.21.15.105:0/994108726 shutdown_connections 2024-09-21T07:26:55.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/994108726 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb8580d0200 0x7fb858152630 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 -- 172.21.15.105:0/994108726 >> 172.21.15.105:0/994108726 conn(0x7fb85806cf80 msgr2=0x7fb85806d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:55.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 -- 172.21.15.105:0/994108726 shutdown_connections 2024-09-21T07:26:55.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.369+0000 7fb85d8d5640 1 -- 172.21.15.105:0/994108726 wait complete. 2024-09-21T07:26:55.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.371+0000 7fb85d8d5640 1 Processor -- start 2024-09-21T07:26:55.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.371+0000 7fb85d8d5640 1 -- start start 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.372+0000 7fb85d8d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8580b0150 0x7fb8580c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.372+0000 7fb85d8d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8580d0200 0x7fb858146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.372+0000 7fb85d8d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb85d8d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb85815e370 con 0x7fb85815c630 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb85d8d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb85815e1f0 con 0x7fb8580d0200 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb85d8d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb85815e4f0 con 0x7fb8580b0150 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38904/0 (socket says 172.21.15.105:38904) 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 -- 172.21.15.105:0/871182816 learned_addr learned my addr 172.21.15.105:0/871182816 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb84ffff640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8580d0200 0x7fb858146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb856ffd640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8580b0150 0x7fb8580c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 -- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8580b0150 msgr2=0x7fb8580c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8580b0150 0x7fb8580c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 -- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8580d0200 msgr2=0x7fb858146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8580d0200 0x7fb858146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 -- 172.21.15.105:0/871182816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8581f37d0 con 0x7fb85815c630 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb8577fe640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b970 tx=0x7fb85000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850019070 con 0x7fb85815c630 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb85000cc30 con 0x7fb85815c630 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500146a0 con 0x7fb85815c630 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8580c6310 con 0x7fb85815c630 2024-09-21T07:26:55.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.373+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8580c68d0 con 0x7fb85815c630 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.374+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8580c6080 con 0x7fb85815c630 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.375+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb85001c070 con 0x7fb85815c630 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.375+0000 7fb854ff9640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 0x7fb83807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.376+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6735+0+0 (secure 0 0 0) 0x7fb85009abc0 con 0x7fb85815c630 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.376+0000 7fb856ffd640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 0x7fb83807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.378 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.376+0000 7fb856ffd640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 0x7fb83807b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb848000f60 tx=0x7fb848009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.382 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.380+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8500626d0 con 0x7fb85815c630 2024-09-21T07:26:55.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.560+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb858152630 con 0x7fb85815c630 2024-09-21T07:26:55.566 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.564+0000 7fb854ff9640 1 -- 172.21.15.105:0/871182816 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fb850067100 con 0x7fb85815c630 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 msgr2=0x7fb83807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 0x7fb83807b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb848000f60 tx=0x7fb848009040 comp rx=0 tx=0).stop 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 msgr2=0x7fb858070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb85000b970 tx=0x7fb85000be30 comp rx=0 tx=0).stop 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 shutdown_connections 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fb838078b90 0x7fb83807b0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8580d0200 0x7fb858146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb85815c630 0x7fb858070990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 --2- 172.21.15.105:0/871182816 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8580b0150 0x7fb8580c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.566+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 >> 172.21.15.105:0/871182816 conn(0x7fb85806cf80 msgr2=0x7fb85815f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:55.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.567+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 shutdown_connections 2024-09-21T07:26:55.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.567+0000 7fb85d8d5640 1 -- 172.21.15.105:0/871182816 wait complete. 2024-09-21T07:26:55.579 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:26:55.579 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:26:55.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:26:55.666 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:26:55.667 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:55.667 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:55.683 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:55.683 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:55.738 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:55.738 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:55.793 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:55.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 -- 172.21.15.105:0/3352572058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00b7940 msgr2=0x7fa8a00b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/3352572058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00b7940 0x7fa8a00b17d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa8a00b74b0 tx=0x7fa89c02a530 comp rx=0 tx=0).stop 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 -- 172.21.15.105:0/3352572058 shutdown_connections 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/3352572058 >> v2:172.21.15.112:3300/0 conn(0x7fa8a00d0310 0x7fa8a00ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/3352572058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00b7940 0x7fa8a00b17d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 -- 172.21.15.105:0/3352572058 >> 172.21.15.105:0/3352572058 conn(0x7fa8a00cedb0 msgr2=0x7fa8a00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 -- 172.21.15.105:0/3352572058 shutdown_connections 2024-09-21T07:26:55.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.947+0000 7fa8a7d83640 1 -- 172.21.15.105:0/3352572058 wait complete. 2024-09-21T07:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.949+0000 7fa8a7d83640 1 Processor -- start 2024-09-21T07:26:55.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.949+0000 7fa8a7d83640 1 -- start start 2024-09-21T07:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa8a00b7940 0x7fa8a015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a00cff00 0x7fa8a009df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a00b1d10 con 0x7fa8a00d0310 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a00cfd40 con 0x7fa8a00cff00 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.950+0000 7fa8a7d83640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a00b1e90 con 0x7fa8a00b7940 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38924/0 (socket says 172.21.15.105:38924) 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 -- 172.21.15.105:0/4246585504 learned_addr learned my addr 172.21.15.105:0/4246585504 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a531e640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a00cff00 0x7fa8a009df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a5b1f640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa8a00b7940 0x7fa8a015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 -- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa8a00b7940 msgr2=0x7fa8a015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa8a00b7940 0x7fa8a015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 -- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a00cff00 msgr2=0x7fa8a009df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:55.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a00cff00 0x7fa8a009df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 -- 172.21.15.105:0/4246585504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a009e9d0 con 0x7fa8a00d0310 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a6320640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa89800ba40 tx=0x7fa89800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898019070 con 0x7fa8a00d0310 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa89800cc90 con 0x7fa8a00d0310 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a009ecb0 con 0x7fa8a00d0310 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8980146a0 con 0x7fa8a00d0310 2024-09-21T07:26:55.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.951+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a009f270 con 0x7fa8a00d0310 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.952+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a00b17d0 con 0x7fa8a00d0310 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.953+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa89801c070 con 0x7fa8a00d0310 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.953+0000 7fa8967fc640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 0x7fa88407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.954+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fa89809ac70 con 0x7fa8a00d0310 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.954+0000 7fa8a5b1f640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 0x7fa88407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:55.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.954+0000 7fa8a5b1f640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 0x7fa88407b0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa89c034000 tx=0x7fa89c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:55.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:55.958+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8980626f0 con 0x7fa8a00d0310 2024-09-21T07:26:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.146+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8a00b2fb0 con 0x7fa8a00d0310 2024-09-21T07:26:56.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.147+0000 7fa8967fc640 1 -- 172.21.15.105:0/4246585504 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fa898067120 con 0x7fa8a00d0310 2024-09-21T07:26:56.149 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 msgr2=0x7fa88407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 0x7fa88407b0d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa89c034000 tx=0x7fa89c004730 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 msgr2=0x7fa8a01f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa89800ba40 tx=0x7fa89800bf00 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 shutdown_connections 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fa884078bc0 0x7fa88407b0d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8a00cff00 0x7fa8a009df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa8a00d0310 0x7fa8a01f7650 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 --2- 172.21.15.105:0/4246585504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa8a00b7940 0x7fa8a015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 >> 172.21.15.105:0/4246585504 conn(0x7fa8a00cedb0 msgr2=0x7fa8a015c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 shutdown_connections 2024-09-21T07:26:56.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.149+0000 7fa8a7d83640 1 -- 172.21.15.105:0/4246585504 wait complete. 2024-09-21T07:26:56.162 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:26:56.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:56.162 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T07:26:56.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:26:56.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:56.188 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:26:56.244 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:56.244 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.264 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:56.264 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.322 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:56.323 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 -- 172.21.15.105:0/3284292492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8308003a80 msgr2=0x7f8308003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 --2- 172.21.15.105:0/3284292492 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8308003a80 0x7f8308003f10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f831000c260 tx=0x7f8308005830 comp rx=0 tx=0).stop 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f8324c96640 1 -- 172.21.15.105:0/3284292492 reap_dead start 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 -- 172.21.15.105:0/3284292492 shutdown_connections 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 -- 172.21.15.105:0/3284292492 >> 172.21.15.105:0/3284292492 conn(0x7f83200cedb0 msgr2=0x7f83200cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 -- 172.21.15.105:0/3284292492 shutdown_connections 2024-09-21T07:26:56.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.494+0000 7f83266f9640 1 -- 172.21.15.105:0/3284292492 wait complete. 2024-09-21T07:26:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.496+0000 7f83266f9640 1 Processor -- start 2024-09-21T07:26:56.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.496+0000 7f83266f9640 1 -- start start 2024-09-21T07:26:56.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.497+0000 7f83266f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 0x7f83200cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f83266f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f83266f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83201f9110 0x7f83201f9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f83266f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f83200cfa10 con 0x7f832009c210 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f83266f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83200b36e0 con 0x7f8308003a80 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f83266f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f83200ccfe0 con 0x7f83201f9110 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38942/0 (socket says 172.21.15.105:38942) 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 -- 172.21.15.105:0/2815676060 learned_addr learned my addr 172.21.15.105:0/2815676060 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831ffff640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 0x7f83200cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f8324c96640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83201f9110 0x7f83201f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 -- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83201f9110 msgr2=0x7f83201f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83201f9110 0x7f83201f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 -- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 msgr2=0x7f83200cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 0x7f83200cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 -- 172.21.15.105:0/2815676060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83201f9a60 con 0x7f832009c210 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831ffff640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 0x7f83200cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831f7fe640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f831400b5b0 tx=0x7f831400ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010a20 con 0x7f832009c210 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.498+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8314015070 con 0x7f832009c210 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.499+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314010e40 con 0x7f832009c210 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.499+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832009f380 con 0x7f832009c210 2024-09-21T07:26:56.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.499+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832009f940 con 0x7f832009c210 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.500+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83200d00e0 con 0x7f832009c210 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.500+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f831401a470 con 0x7f832009c210 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.501+0000 7f831cff9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 0x7f830c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.501+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f831409ba70 con 0x7f832009c210 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.501+0000 7f831ffff640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 0x7f830c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:56.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.502+0000 7f831ffff640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 0x7f830c07b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8310033000 tx=0x7f8310038040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:56.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.506+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83201f9a60 con 0x7f832009c210 2024-09-21T07:26:56.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.695+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83200b3c00 con 0x7f832009c210 2024-09-21T07:26:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.696+0000 7f831cff9640 1 -- 172.21.15.105:0/2815676060 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f83140634f0 con 0x7f832009c210 2024-09-21T07:26:56.698 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 msgr2=0x7f830c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 0x7f830c07b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8310033000 tx=0x7f8310038040 comp rx=0 tx=0).stop 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 msgr2=0x7f832009ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f831400b5b0 tx=0x7f831400ba70 comp rx=0 tx=0).stop 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 shutdown_connections 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f830c078bc0 0x7f830c07b0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83201f9110 0x7f83201f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f832009c210 0x7f832009ea70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 --2- 172.21.15.105:0/2815676060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8308003a80 0x7f83200cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 >> 172.21.15.105:0/2815676060 conn(0x7f83200cedb0 msgr2=0x7f83200b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 shutdown_connections 2024-09-21T07:26:56.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:56.698+0000 7f83266f9640 1 -- 172.21.15.105:0/2815676060 wait complete. 2024-09-21T07:26:56.711 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:26:56.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:56.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T07:26:56.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:26:56.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:56.743 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:26:56.815 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:56.815 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.832 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:56.832 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.888 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:26:56.888 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:56.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:57.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 -- 172.21.15.105:0/1083565871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40b1a90 msgr2=0x7f17c40b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 --2- 172.21.15.105:0/1083565871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40b1a90 0x7f17c40b1ea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009900 tx=0x7f17ac02a580 comp rx=0 tx=0).stop 2024-09-21T07:26:57.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 -- 172.21.15.105:0/1083565871 shutdown_connections 2024-09-21T07:26:57.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 --2- 172.21.15.105:0/1083565871 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40b1a90 0x7f17c40b1ea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 -- 172.21.15.105:0/1083565871 >> 172.21.15.105:0/1083565871 conn(0x7f17c40cee40 msgr2=0x7f17c40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:57.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 -- 172.21.15.105:0/1083565871 shutdown_connections 2024-09-21T07:26:57.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.095+0000 7f17caebd640 1 -- 172.21.15.105:0/1083565871 wait complete. 2024-09-21T07:26:57.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.097+0000 7f17caebd640 1 Processor -- start 2024-09-21T07:26:57.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.097+0000 7f17caebd640 1 -- start start 2024-09-21T07:26:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 0x7f17c415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c40ce300 0x7f17c4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4167cb0 con 0x7f17c40c5a50 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17c409bc40 con 0x7f17c40b1a90 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17caebd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f17c4167e30 con 0x7f17c40ce300 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.098+0000 7f17c3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38946/0 (socket says 172.21.15.105:38946) 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 -- 172.21.15.105:0/3114232476 learned_addr learned my addr 172.21.15.105:0/3114232476 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c8c59640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 0x7f17c415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 -- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c40ce300 msgr2=0x7f17c4157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c40ce300 0x7f17c4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 -- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 msgr2=0x7f17c415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 0x7f17c415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.102 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 -- 172.21.15.105:0/3114232476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c40aefb0 con 0x7f17c40c5a50 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c8c59640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 0x7f17c415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c3fff640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f17ac000c00 tx=0x7f17ac0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac03e030 con 0x7f17c40c5a50 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17ac0047a0 con 0x7f17c40c5a50 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c4158f50 con 0x7f17c40c5a50 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac035040 con 0x7f17c40c5a50 2024-09-21T07:26:57.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.099+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41f6fe0 con 0x7f17c40c5a50 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.100+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c41f6a70 con 0x7f17c40c5a50 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.101+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17ac041030 con 0x7f17c40c5a50 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.101+0000 7f17c17fa640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 0x7f17a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.102+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f17ac0bba00 con 0x7f17c40c5a50 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.102+0000 7f17c8c59640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 0x7f17a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:57.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.102+0000 7f17c8c59640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 0x7f17a407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17b4007e90 tx=0x7f17b40073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:57.107 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.106+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17ac083ad0 con 0x7f17c40c5a50 2024-09-21T07:26:57.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.292+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f17c40b1ea0 con 0x7f17c40c5a50 2024-09-21T07:26:57.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.293+0000 7f17c17fa640 1 -- 172.21.15.105:0/3114232476 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f17ac0833d0 con 0x7f17c40c5a50 2024-09-21T07:26:57.295 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 msgr2=0x7f17a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 0x7f17a407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f17b4007e90 tx=0x7f17b40073a0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 msgr2=0x7f17c4156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f17ac000c00 tx=0x7f17ac0039e0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 shutdown_connections 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f17a4078b90 0x7f17a407b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17c40ce300 0x7f17c4157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17c40c5a50 0x7f17c4156390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 --2- 172.21.15.105:0/3114232476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17c40b1a90 0x7f17c415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 >> 172.21.15.105:0/3114232476 conn(0x7f17c40cee40 msgr2=0x7f17c40ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 shutdown_connections 2024-09-21T07:26:57.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.295+0000 7f17caebd640 1 -- 172.21.15.105:0/3114232476 wait complete. 2024-09-21T07:26:57.308 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T07:26:57.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:57.309 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T07:26:57.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:26:57.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:57.335 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T07:26:57.392 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:57.392 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:57.411 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:57.411 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:57.470 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:26:57.471 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:26:57.528 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 -- 172.21.15.105:0/85937559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0cc1d0 msgr2=0x7ff06c073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 --2- 172.21.15.105:0/85937559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0cc1d0 0x7ff06c073dc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff06c073370 tx=0x7ff06002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 -- 172.21.15.105:0/85937559 shutdown_connections 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 --2- 172.21.15.105:0/85937559 >> v2:172.21.15.112:3300/0 conn(0x7ff06c0ca760 0x7ff06c0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 --2- 172.21.15.105:0/85937559 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0cc1d0 0x7ff06c073dc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 -- 172.21.15.105:0/85937559 >> 172.21.15.105:0/85937559 conn(0x7ff06c0ccb90 msgr2=0x7ff06c0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:57.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 -- 172.21.15.105:0/85937559 shutdown_connections 2024-09-21T07:26:57.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.640+0000 7ff070950640 1 -- 172.21.15.105:0/85937559 wait complete. 2024-09-21T07:26:57.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.642+0000 7ff070950640 1 Processor -- start 2024-09-21T07:26:57.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.642+0000 7ff070950640 1 -- start start 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c0cb4a0 0x7ff06c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff06c0b4450 0x7ff06c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c0cea80 con 0x7ff06c0ca760 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c0ce900 con 0x7ff06c0cb4a0 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff070950640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff06c0cec00 con 0x7ff06c0b4450 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff06a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:57.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff06a59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38962/0 (socket says 172.21.15.105:38962) 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.643+0000 7ff06a59c640 1 -- 172.21.15.105:0/2960038136 learned_addr learned my addr 172.21.15.105:0/2960038136 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 -- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff06c0b4450 msgr2=0x7ff06c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff06c0b4450 0x7ff06c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 -- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c0cb4a0 msgr2=0x7ff06c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c0cb4a0 0x7ff06c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 -- 172.21.15.105:0/2960038136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c09cdb0 con 0x7ff06c0ca760 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff06a59c640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c3c0 tx=0x7ff05c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c017070 con 0x7ff06c0ca760 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff05c0035f0 con 0x7ff06c0ca760 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c09d090 con 0x7ff06c0ca760 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c003a80 con 0x7ff06c0ca760 2024-09-21T07:26:57.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.644+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c09d650 con 0x7ff06c0ca760 2024-09-21T07:26:57.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.645+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06c0cf120 con 0x7ff06c0ca760 2024-09-21T07:26:57.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.646+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff05c01a070 con 0x7ff06c0ca760 2024-09-21T07:26:57.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.646+0000 7ff05affd640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 0x7ff04c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:57.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.647+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7ff05c09a4b0 con 0x7ff06c0ca760 2024-09-21T07:26:57.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.647+0000 7ff069d9b640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 0x7ff04c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:57.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.647+0000 7ff069d9b640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 0x7ff04c07b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff060034000 tx=0x7ff0600046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.651+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff05c0624f0 con 0x7ff06c0ca760 2024-09-21T07:26:57.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.837+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff06c0ae560 con 0x7ff06c0ca760 2024-09-21T07:26:57.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.837+0000 7ff05affd640 1 -- 172.21.15.105:0/2960038136 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7ff05c061df0 con 0x7ff06c0ca760 2024-09-21T07:26:57.839 INFO:teuthology.orchestra.run.smithi105.stderr:listed 0 entries 2024-09-21T07:26:57.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 msgr2=0x7ff04c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.841 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 0x7ff04c07b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff060034000 tx=0x7ff0600046c0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 msgr2=0x7ff06c0c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c3c0 tx=0x7ff05c00c880 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 shutdown_connections 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7ff04c078b90 0x7ff04c07b0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff06c0cb4a0 0x7ff06c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff06c0b4450 0x7ff06c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 --2- 172.21.15.105:0/2960038136 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff06c0ca760 0x7ff06c0c69d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 >> 172.21.15.105:0/2960038136 conn(0x7ff06c0ccb90 msgr2=0x7ff06c15e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 shutdown_connections 2024-09-21T07:26:57.842 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:57.839+0000 7ff070950640 1 -- 172.21.15.105:0/2960038136 wait complete. 2024-09-21T07:26:57.852 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T07:26:57.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:57.853 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T07:26:57.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:26:57.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:26:57.884 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T07:26:57.942 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:26:57.943 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 -- 172.21.15.105:0/206599626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180d06b0 msgr2=0x7f04180b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 --2- 172.21.15.105:0/206599626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180d06b0 0x7f04180b1a90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0408009830 tx=0x7f040802a480 comp rx=0 tx=0).stop 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 -- 172.21.15.105:0/206599626 shutdown_connections 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 --2- 172.21.15.105:0/206599626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180d06b0 0x7f04180b1a90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 --2- 172.21.15.105:0/206599626 >> v2:172.21.15.112:3300/0 conn(0x7f04180c5a50 0x7f04180d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.053+0000 7f041f982640 1 -- 172.21.15.105:0/206599626 >> 172.21.15.105:0/206599626 conn(0x7f04180cee40 msgr2=0x7f04180cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.054+0000 7f041f982640 1 -- 172.21.15.105:0/206599626 shutdown_connections 2024-09-21T07:26:58.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.054+0000 7f041f982640 1 -- 172.21.15.105:0/206599626 wait complete. 2024-09-21T07:26:58.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.055+0000 7f041f982640 1 Processor -- start 2024-09-21T07:26:58.057 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.055+0000 7f041f982640 1 -- start start 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180c5a50 0x7f041814c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180d06b0 0x7f04181f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f04180ccfb0 con 0x7f04180b1fd0 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04180cec60 con 0x7f04180d06b0 2024-09-21T07:26:58.060 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f04180cd130 con 0x7f04180c5a50 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38978/0 (socket says 172.21.15.105:38978) 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 -- 172.21.15.105:0/2509668504 learned_addr learned my addr 172.21.15.105:0/2509668504 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041cf1d640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180c5a50 0x7f041814c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041df1f640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180d06b0 0x7f04181f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 -- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180c5a50 msgr2=0x7f041814c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180c5a50 0x7f041814c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 -- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180d06b0 msgr2=0x7f04181f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180d06b0 0x7f04181f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 -- 172.21.15.105:0/2509668504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041814c910 con 0x7f04180b1fd0 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041d71e640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f041400ea20 tx=0x7f041400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.061 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041400ce40 con 0x7f04180b1fd0 2024-09-21T07:26:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0414004020 con 0x7f04180b1fd0 2024-09-21T07:26:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04180cbc80 con 0x7f04180b1fd0 2024-09-21T07:26:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.057+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414004470 con 0x7f04180b1fd0 2024-09-21T07:26:58.062 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.058+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04180cc240 con 0x7f04180b1fd0 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.059+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04180d0170 con 0x7f04180b1fd0 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.059+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0414005030 con 0x7f04180b1fd0 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.060+0000 7f040dffb640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 0x7f03fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.060+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f041409b8d0 con 0x7f04180b1fd0 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.060+0000 7f041cf1d640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 0x7f03fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.063 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.061+0000 7f041cf1d640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 0x7f03fc07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0408034000 tx=0x7f0408004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.065+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04140632d0 con 0x7f04180b1fd0 2024-09-21T07:26:58.277 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.275+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04180d1100 con 0x7f04180b1fd0 2024-09-21T07:26:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.276+0000 7f040dffb640 1 -- 172.21.15.105:0/2509668504 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f0414099e20 con 0x7f04180b1fd0 2024-09-21T07:26:58.278 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:58.278 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 msgr2=0x7f03fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 0x7f03fc07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0408034000 tx=0x7f0408004730 comp rx=0 tx=0).stop 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 msgr2=0x7f04181f2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f041400ea20 tx=0x7f041400eee0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 shutdown_connections 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f03fc078bc0 0x7f03fc07b0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180d06b0 0x7f04181f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f04180c5a50 0x7f041814c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 --2- 172.21.15.105:0/2509668504 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f04180b1fd0 0x7f04181f2fc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 >> 172.21.15.105:0/2509668504 conn(0x7f04180cee40 msgr2=0x7f04180b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 shutdown_connections 2024-09-21T07:26:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.278+0000 7f041f982640 1 -- 172.21.15.105:0/2509668504 wait complete. 2024-09-21T07:26:58.291 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:58.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 -- 172.21.15.105:0/3729688030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 msgr2=0x7f5a2c06f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 --2- 172.21.15.105:0/3729688030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c06f7f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c06ff60 tx=0x7f5a1c02a510 comp rx=0 tx=0).stop 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 -- 172.21.15.105:0/3729688030 shutdown_connections 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 --2- 172.21.15.105:0/3729688030 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c06f7f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 -- 172.21.15.105:0/3729688030 >> 172.21.15.105:0/3729688030 conn(0x7f5a2c08bfb0 msgr2=0x7f5a2c0d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 -- 172.21.15.105:0/3729688030 shutdown_connections 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.402+0000 7f5a32589640 1 -- 172.21.15.105:0/3729688030 wait complete. 2024-09-21T07:26:58.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.404+0000 7f5a32589640 1 Processor -- start 2024-09-21T07:26:58.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.404+0000 7f5a32589640 1 -- start start 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 0x7f5a2c1f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2c0b6850 0x7f5a2c1f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c169f30 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c0b3b60 con 0x7f5a2c0b6440 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a32589640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5a2c16a0b0 con 0x7f5a2c0b6850 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.405+0000 7f5a2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38994/0 (socket says 172.21.15.105:38994) 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1348748410 learned_addr learned my addr 172.21.15.105:0/1348748410 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:58.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a30b26640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2c0b6850 0x7f5a2c1f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2b7fe640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 0x7f5a2c1f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2c0b6850 msgr2=0x7f5a2c1f91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2c0b6850 0x7f5a2c1f91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 msgr2=0x7f5a2c1f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 0x7f5a2c1f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 -- 172.21.15.105:0/1348748410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c0b9630 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2b7fe640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 0x7f5a2c1f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a2bfff640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098f0 tx=0x7f5a1c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c005c90 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a1c031e50 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c0c9a00 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c005260 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.406+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c0c9cc0 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.407+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2c0d18b0 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.408+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a1c005400 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.408+0000 7f5a28ff9640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 0x7f5a1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.409+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f5a1c0bc2d0 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.409+0000 7f5a2b7fe640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 0x7f5a1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.409+0000 7f5a2b7fe640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 0x7f5a1807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c1fa2c0 tx=0x7f5a20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.413+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a2c0d18b0 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.624+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a2c050f60 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.625+0000 7f5a28ff9640 1 -- 172.21.15.105:0/1348748410 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f5a2c050f60 con 0x7f5a2c08c3c0 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:58.627 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:58.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 msgr2=0x7f5a1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 0x7f5a1807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c1fa2c0 tx=0x7f5a20009040 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 msgr2=0x7f5a2c0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0098f0 tx=0x7f5a1c033c20 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 shutdown_connections 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5a18078b90 0x7f5a1807b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c0b6440 0x7f5a2c1f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a2c0b6850 0x7f5a2c1f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 --2- 172.21.15.105:0/1348748410 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5a2c08c3c0 0x7f5a2c0ca9a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.627+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 >> 172.21.15.105:0/1348748410 conn(0x7f5a2c08bfb0 msgr2=0x7f5a2c0d11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.628+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 shutdown_connections 2024-09-21T07:26:58.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.628+0000 7f5a32589640 1 -- 172.21.15.105:0/1348748410 wait complete. 2024-09-21T07:26:58.640 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:58.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 -- 172.21.15.105:0/2913410798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840d0a30 msgr2=0x7f0b840d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/2913410798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840d0a30 0x7f0b840d0e40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0b840b72c0 tx=0x7f0b7c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 -- 172.21.15.105:0/2913410798 shutdown_connections 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/2913410798 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840d0a30 0x7f0b840d0e40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/2913410798 >> v2:172.21.15.112:3300/0 conn(0x7f0b840cff00 0x7f0b840d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 -- 172.21.15.105:0/2913410798 >> 172.21.15.105:0/2913410798 conn(0x7f0b840cedb0 msgr2=0x7f0b840cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 -- 172.21.15.105:0/2913410798 shutdown_connections 2024-09-21T07:26:58.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.751+0000 7f0b88e0e640 1 -- 172.21.15.105:0/2913410798 wait complete. 2024-09-21T07:26:58.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.753+0000 7f0b88e0e640 1 Processor -- start 2024-09-21T07:26:58.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.753+0000 7f0b88e0e640 1 -- start start 2024-09-21T07:26:58.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b840b7940 0x7f0b8415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b840d0a30 0x7f0b840afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0b840cebe0 con 0x7f0b840cff00 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b840cea60 con 0x7f0b840d0a30 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.754+0000 7f0b88e0e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0b840ccf40 con 0x7f0b840b7940 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39022/0 (socket says 172.21.15.105:39022) 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 -- 172.21.15.105:0/425348273 learned_addr learned my addr 172.21.15.105:0/425348273 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b82d9d640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b840d0a30 0x7f0b840afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b8259c640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b840b7940 0x7f0b8415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 -- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b840b7940 msgr2=0x7f0b8415aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b840b7940 0x7f0b8415aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 -- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b840d0a30 msgr2=0x7f0b840afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b840d0a30 0x7f0b840afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 -- 172.21.15.105:0/425348273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b841f5490 con 0x7f0b840cff00 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b81d9b640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d750 tx=0x7f0b7800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78017070 con 0x7f0b840cff00 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b78004310 con 0x7f0b840cff00 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8415f0d0 con 0x7f0b840cff00 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78009c30 con 0x7f0b840cff00 2024-09-21T07:26:58.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.755+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8415e7a0 con 0x7f0b840cff00 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.756+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b840d0310 con 0x7f0b840cff00 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.757+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b78005020 con 0x7f0b840cff00 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.758+0000 7f0b72ffd640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 0x7f0b6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.758+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f0b7809b8d0 con 0x7f0b840cff00 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.758+0000 7f0b8259c640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 0x7f0b6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:58.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.758+0000 7f0b8259c640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 0x7f0b6407b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b74001720 tx=0x7f0b74007e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:58.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.761+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b78020020 con 0x7f0b840cff00 2024-09-21T07:26:58.943 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.941+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b840b3c00 con 0x7f0b840cff00 2024-09-21T07:26:58.944 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.942+0000 7f0b72ffd640 1 -- 172.21.15.105:0/425348273 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+19057 (secure 0 0 0) 0x7f0b7800de60 con 0x7f0b840cff00 2024-09-21T07:26:58.944 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:58.945 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":29,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:55.920783+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:58.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 msgr2=0x7f0b6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 0x7f0b6407b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0b74001720 tx=0x7f0b74007e90 comp rx=0 tx=0).stop 2024-09-21T07:26:58.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 msgr2=0x7f0b8415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:58.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800d750 tx=0x7f0b7800dc10 comp rx=0 tx=0).stop 2024-09-21T07:26:58.948 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 shutdown_connections 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0b64078b90 0x7f0b6407b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b840d0a30 0x7f0b840afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b840cff00 0x7f0b8415a370 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 --2- 172.21.15.105:0/425348273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0b840b7940 0x7f0b8415aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 >> 172.21.15.105:0/425348273 conn(0x7f0b840cedb0 msgr2=0x7f0b8415cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 shutdown_connections 2024-09-21T07:26:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:58.945+0000 7f0b88e0e640 1 -- 172.21.15.105:0/425348273 wait complete. 2024-09-21T07:26:58.959 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:26:58.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 -- 172.21.15.105:0/1124685928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf28004d40 msgr2=0x7faf280051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 --2- 172.21.15.105:0/1124685928 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf28004d40 0x7faf280051f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7faf34001d90 tx=0x7faf28005830 comp rx=0 tx=0).stop 2024-09-21T07:26:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf42228640 1 -- 172.21.15.105:0/1124685928 reap_dead start 2024-09-21T07:26:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 -- 172.21.15.105:0/1124685928 shutdown_connections 2024-09-21T07:26:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 -- 172.21.15.105:0/1124685928 >> 172.21.15.105:0/1124685928 conn(0x7faf3c0cee20 msgr2=0x7faf3c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 -- 172.21.15.105:0/1124685928 shutdown_connections 2024-09-21T07:26:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.070+0000 7faf43c8b640 1 -- 172.21.15.105:0/1124685928 wait complete. 2024-09-21T07:26:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.072+0000 7faf43c8b640 1 Processor -- start 2024-09-21T07:26:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.072+0000 7faf43c8b640 1 -- start start 2024-09-21T07:26:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf3c151120 0x7faf3c072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 0x7faf3c1f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c075330 con 0x7faf3c1f6020 2024-09-21T07:26:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c0b74a0 con 0x7faf3c151120 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf43c8b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c15cdd0 con 0x7faf3c072770 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf41a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf41a27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39040/0 (socket says 172.21.15.105:39040) 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.073+0000 7faf41a27640 1 -- 172.21.15.105:0/513365893 learned_addr learned my addr 172.21.15.105:0/513365893 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf42228640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 0x7faf3c1f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41226640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf3c151120 0x7faf3c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 -- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 msgr2=0x7faf3c1f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 0x7faf3c1f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 -- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf3c151120 msgr2=0x7faf3c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf3c151120 0x7faf3c072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 -- 172.21.15.105:0/513365893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c072b80 con 0x7faf3c1f6020 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf42228640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 0x7faf3c1f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf41a27640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf38003670 tx=0x7faf38003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38015070 con 0x7faf3c1f6020 2024-09-21T07:26:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf38011900 con 0x7faf3c1f6020 2024-09-21T07:26:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c162940 con 0x7faf3c1f6020 2024-09-21T07:26:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf38011d50 con 0x7faf3c1f6020 2024-09-21T07:26:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.074+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c162660 con 0x7faf3c1f6020 2024-09-21T07:26:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.075+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c0b1e30 con 0x7faf3c1f6020 2024-09-21T07:26:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.076+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf38018070 con 0x7faf3c1f6020 2024-09-21T07:26:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.076+0000 7faf327fc640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 0x7faf2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.076+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7faf3809be80 con 0x7faf3c1f6020 2024-09-21T07:26:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.077+0000 7faf41226640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 0x7faf2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.077+0000 7faf41226640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 0x7faf2007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf3c077200 tx=0x7faf2c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.081+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf3c072b80 con 0x7faf3c1f6020 2024-09-21T07:26:59.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.291+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf3c0afdd0 con 0x7faf3c1f6020 2024-09-21T07:26:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.292+0000 7faf327fc640 1 -- 172.21.15.105:0/513365893 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7faf38063fe0 con 0x7faf3c1f6020 2024-09-21T07:26:59.294 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:59.294 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 msgr2=0x7faf2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 0x7faf2007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faf3c077200 tx=0x7faf2c005f90 comp rx=0 tx=0).stop 2024-09-21T07:26:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 msgr2=0x7faf3c1518b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf38003670 tx=0x7faf38003aa0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 shutdown_connections 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7faf20078b90 0x7faf2007b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf3c151120 0x7faf3c072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf3c072770 0x7faf3c1f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 --2- 172.21.15.105:0/513365893 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf3c1f6020 0x7faf3c1518b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 >> 172.21.15.105:0/513365893 conn(0x7faf3c0cee20 msgr2=0x7faf3c071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 shutdown_connections 2024-09-21T07:26:59.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.294+0000 7faf43c8b640 1 -- 172.21.15.105:0/513365893 wait complete. 2024-09-21T07:26:59.307 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 -- 172.21.15.105:0/1482025788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80b1fd0 msgr2=0x7f05e80ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 --2- 172.21.15.105:0/1482025788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80b1fd0 0x7f05e80ce300 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05e80768e0 tx=0x7f05e002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 -- 172.21.15.105:0/1482025788 shutdown_connections 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 --2- 172.21.15.105:0/1482025788 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80b1fd0 0x7f05e80ce300 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 --2- 172.21.15.105:0/1482025788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80d06b0 0x7f05e80b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 --2- 172.21.15.105:0/1482025788 >> v2:172.21.15.112:3300/0 conn(0x7f05e80c5a50 0x7f05e80d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 -- 172.21.15.105:0/1482025788 >> 172.21.15.105:0/1482025788 conn(0x7f05e80cee40 msgr2=0x7f05e80cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 -- 172.21.15.105:0/1482025788 shutdown_connections 2024-09-21T07:26:59.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.417+0000 7f05ed620640 1 -- 172.21.15.105:0/1482025788 wait complete. 2024-09-21T07:26:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.419+0000 7f05ed620640 1 Processor -- start 2024-09-21T07:26:59.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.419+0000 7f05ed620640 1 -- start start 2024-09-21T07:26:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80b1fd0 0x7f05e815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e80c5a50 0x7f05e80b67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05e80cebd0 con 0x7f05e80d06b0 2024-09-21T07:26:59.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05e80cea50 con 0x7f05e80c5a50 2024-09-21T07:26:59.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05ed620640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f05e809bc80 con 0x7f05e80b1fd0 2024-09-21T07:26:59.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.420+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39052/0 (socket says 172.21.15.105:39052) 2024-09-21T07:26:59.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 -- 172.21.15.105:0/472916324 learned_addr learned my addr 172.21.15.105:0/472916324 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e67fc640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e80c5a50 0x7f05e80b67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e6ffd640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80b1fd0 0x7f05e815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 -- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80b1fd0 msgr2=0x7f05e815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80b1fd0 0x7f05e815cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 -- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e80c5a50 msgr2=0x7f05e80b67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e80c5a50 0x7f05e80b67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 -- 172.21.15.105:0/472916324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e81f3fa0 con 0x7f05e80d06b0 2024-09-21T07:26:59.424 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05e77fe640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099c0 tx=0x7f05e0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e003e030 con 0x7f05e80d06b0 2024-09-21T07:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e0003e40 con 0x7f05e80d06b0 2024-09-21T07:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e81f4280 con 0x7f05e80d06b0 2024-09-21T07:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0035040 con 0x7f05e80d06b0 2024-09-21T07:26:59.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.421+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e80cb0a0 con 0x7f05e80d06b0 2024-09-21T07:26:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.422+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e81f3d10 con 0x7f05e80d06b0 2024-09-21T07:26:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.423+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05e0041030 con 0x7f05e80d06b0 2024-09-21T07:26:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.423+0000 7f05cf7fe640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 0x7f05d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.423+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f05e00bb850 con 0x7f05e80d06b0 2024-09-21T07:26:59.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.424+0000 7f05e6ffd640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 0x7f05d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.427 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.424+0000 7f05e6ffd640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 0x7f05d407b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005fd0 tx=0x7f05d80095b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.428+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05e0083250 con 0x7f05e80d06b0 2024-09-21T07:26:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.642+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e8072090 con 0x7f05e80d06b0 2024-09-21T07:26:59.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.643+0000 7f05cf7fe640 1 -- 172.21.15.105:0/472916324 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f05e8072090 con 0x7f05e80d06b0 2024-09-21T07:26:59.645 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:59.645 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":9,"btime":"2024-09-21T07:26:35:827189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3610578586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3610578586},{"type":"v1","addr":"172.21.15.112:6837","nonce":3610578586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/3018650664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6833","nonce":3018650664},{"type":"v1","addr":"172.21.15.105:6835","nonce":3018650664}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4422,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6834/1705149967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1705149967},{"type":"v1","addr":"172.21.15.105:6834","nonce":1705149967}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:26:15.095198+0000","modified":"2024-09-21T07:26:35.182301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4434":{"gid":4434,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/3289542241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3289542241},{"type":"v1","addr":"172.21.15.112:6836","nonce":3289542241}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4434,"qdb_cluster":[4434]},"id":1}]} 2024-09-21T07:26:59.645 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 9 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 msgr2=0x7f05d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 0x7f05d407b0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005fd0 tx=0x7f05d80095b0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 msgr2=0x7f05e80b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f05e00099c0 tx=0x7f05e0003910 comp rx=0 tx=0).stop 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 shutdown_connections 2024-09-21T07:26:59.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d4078b90 0x7f05d407b0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e80c5a50 0x7f05e80b67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e80d06b0 0x7f05e80b6d20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 --2- 172.21.15.105:0/472916324 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e80b1fd0 0x7f05e815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 >> 172.21.15.105:0/472916324 conn(0x7f05e80cee40 msgr2=0x7f05e80ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 shutdown_connections 2024-09-21T07:26:59.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.645+0000 7f05ed620640 1 -- 172.21.15.105:0/472916324 wait complete. 2024-09-21T07:26:59.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:26:59.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 -- 172.21.15.105:0/3371681319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 msgr2=0x7f15a00703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 --2- 172.21.15.105:0/3371681319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a00703e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15a00cf750 tx=0x7f159802a980 comp rx=0 tx=0).stop 2024-09-21T07:26:59.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 -- 172.21.15.105:0/3371681319 shutdown_connections 2024-09-21T07:26:59.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 --2- 172.21.15.105:0/3371681319 >> v2:172.21.15.112:3300/0 conn(0x7f15a00744f0 0x7f15a00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 --2- 172.21.15.105:0/3371681319 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a00703e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 -- 172.21.15.105:0/3371681319 >> 172.21.15.105:0/3371681319 conn(0x7f15a00cdce0 msgr2=0x7f15a00ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 -- 172.21.15.105:0/3371681319 shutdown_connections 2024-09-21T07:26:59.772 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.768+0000 7f15a4f19640 1 -- 172.21.15.105:0/3371681319 wait complete. 2024-09-21T07:26:59.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.770+0000 7f15a4f19640 1 Processor -- start 2024-09-21T07:26:59.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.770+0000 7f15a4f19640 1 -- start start 2024-09-21T07:26:59.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a00744f0 0x7f15a01f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a00d0d90 0x7f15a01f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.777 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0160940 con 0x7f15a006ffd0 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0070dd0 con 0x7f15a00d0d90 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15a4f19640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f15a0160ac0 con 0x7f15a00744f0 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39074/0 (socket says 172.21.15.105:39074) 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 -- 172.21.15.105:0/2346624857 learned_addr learned my addr 172.21.15.105:0/2346624857 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:26:59.778 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159ed9d640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a00d0d90 0x7f15a01f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f1595d9b640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a00744f0 0x7f15a01f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 -- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a00744f0 msgr2=0x7f15a01f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a00744f0 0x7f15a01f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 -- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a00d0d90 msgr2=0x7f15a01f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a00d0d90 0x7f15a01f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 -- 172.21.15.105:0/2346624857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a015fd90 con 0x7f15a006ffd0 2024-09-21T07:26:59.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f159e59c640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f159000b420 tx=0x7f159000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.772+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590011830 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.773+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1590016070 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.773+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a015f4f0 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.773+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590011d60 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.773+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a015fab0 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.774+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a00d0010 con 0x7f15a006ffd0 2024-09-21T07:26:59.780 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.774+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f159001b460 con 0x7f15a006ffd0 2024-09-21T07:26:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.775+0000 7f15977fe640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 0x7f158007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:26:59.781 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.775+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7f159009acb0 con 0x7f15a006ffd0 2024-09-21T07:26:59.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.775+0000 7f1595d9b640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 0x7f158007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:26:59.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.776+0000 7f1595d9b640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 0x7f158007b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f158800a490 tx=0x7f158800a420 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:26:59.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.779+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15a00d0010 con 0x7f15a006ffd0 2024-09-21T07:26:59.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.959+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f15a00aff80 con 0x7f15a006ffd0 2024-09-21T07:26:59.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.960+0000 7f15977fe640 1 -- 172.21.15.105:0/2346624857 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+19057 (secure 0 0 0) 0x7f15900626a0 con 0x7f15a006ffd0 2024-09-21T07:26:59.963 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:26:59.963 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":29,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:26:55.920783+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:26:13.030140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:26:14.056565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-21T07:26:17.764268+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.4","mappings":[{"from":1,"to":5}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:26:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 msgr2=0x7f158007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 0x7f158007b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f158800a490 tx=0x7f158800a420 comp rx=0 tx=0).stop 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 msgr2=0x7f15a009d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f159000b420 tx=0x7f159000b8e0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 shutdown_connections 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1580078b90 0x7f158007b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15a00d0d90 0x7f15a01f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15a00744f0 0x7f15a01f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 --2- 172.21.15.105:0/2346624857 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f15a006ffd0 0x7f15a009d7c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:26:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 >> 172.21.15.105:0/2346624857 conn(0x7f15a00cdce0 msgr2=0x7f15a015c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:26:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 shutdown_connections 2024-09-21T07:26:59.967 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:26:59.963+0000 7f15a4f19640 1 -- 172.21.15.105:0/2346624857 wait complete. 2024-09-21T07:26:59.977 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:27:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 -- 172.21.15.105:0/1545317578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc378003a00 msgr2=0x7fc378003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 --2- 172.21.15.105:0/1545317578 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc378003a00 0x7fc378003e70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc38000c2e0 tx=0x7fc378005830 comp rx=0 tx=0).stop 2024-09-21T07:27:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc38ffff640 1 -- 172.21.15.105:0/1545317578 reap_dead start 2024-09-21T07:27:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 -- 172.21.15.105:0/1545317578 shutdown_connections 2024-09-21T07:27:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 -- 172.21.15.105:0/1545317578 >> 172.21.15.105:0/1545317578 conn(0x7fc39009d850 msgr2=0x7fc39008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 -- 172.21.15.105:0/1545317578 shutdown_connections 2024-09-21T07:27:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.089+0000 7fc3960a4640 1 -- 172.21.15.105:0/1545317578 wait complete. 2024-09-21T07:27:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.090+0000 7fc3960a4640 1 Processor -- start 2024-09-21T07:27:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.091+0000 7fc3960a4640 1 -- start start 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc378003a00 0x7fc3901f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 0x7fc3901f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc3900cf2f0 con 0x7fc390151670 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3900cf170 con 0x7fc390151260 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc3960a4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc3900d3ad0 con 0x7fc378003a00 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39088/0 (socket says 172.21.15.105:39088) 2024-09-21T07:27:00.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 -- 172.21.15.105:0/3999226948 learned_addr learned my addr 172.21.15.105:0/3999226948 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38effd640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 0x7fc3901f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38f7fe640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc378003a00 0x7fc3901f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 -- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc378003a00 msgr2=0x7fc3901f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc378003a00 0x7fc3901f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 -- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 msgr2=0x7fc3901f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 0x7fc3901f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 -- 172.21.15.105:0/3999226948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901f61b0 con 0x7fc390151670 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38effd640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 0x7fc3901f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:00.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.092+0000 7fc38ffff640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc3900d0700 tx=0x7fc38800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.093+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388003b80 con 0x7fc390151670 2024-09-21T07:27:00.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.093+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc388003e10 con 0x7fc390151670 2024-09-21T07:27:00.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.093+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388004d80 con 0x7fc390151670 2024-09-21T07:27:00.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.093+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc390151bd0 con 0x7fc390151670 2024-09-21T07:27:00.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.093+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3900cc800 con 0x7fc390151670 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.094+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3900cc620 con 0x7fc390151670 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.094+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc38801a020 con 0x7fc390151670 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.095+0000 7fc377fff640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 0x7fc37c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.095+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6735+0+0 (secure 0 0 0) 0x7fc38809b3d0 con 0x7fc390151670 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.095+0000 7fc38f7fe640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 0x7fc37c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.096+0000 7fc38f7fe640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 0x7fc37c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc3900d2790 tx=0x7fc38002d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.099+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3900cc620 con 0x7fc390151670 2024-09-21T07:27:00.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.314+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc3900d19e0 con 0x7fc390151670 2024-09-21T07:27:00.316 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-09-21T07:27:00.315+0000 7f3c46d8f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:27:00.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.316+0000 7fc377fff640 1 -- 172.21.15.105:0/3999226948 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fc3900d19e0 con 0x7fc390151670 2024-09-21T07:27:00.318 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:27:00.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 msgr2=0x7fc37c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 0x7fc37c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc3900d2790 tx=0x7fc38002d000 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 msgr2=0x7fc3901f8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc3900d0700 tx=0x7fc38800fd90 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 shutdown_connections 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc37c078b90 0x7fc37c07b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc390151260 0x7fc3901f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc390151670 0x7fc3901f8d20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 --2- 172.21.15.105:0/3999226948 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc378003a00 0x7fc3901f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.319+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 >> 172.21.15.105:0/3999226948 conn(0x7fc39009d850 msgr2=0x7fc390068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.320+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 shutdown_connections 2024-09-21T07:27:00.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.320+0000 7fc3960a4640 1 -- 172.21.15.105:0/3999226948 wait complete. 2024-09-21T07:27:00.326 INFO:tasks.ceph.mds.b.smithi112.stderr: -4531> 2024-09-21T07:26:20.712+0000 7fb2d985c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:27:00.333 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:27:00.383 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-09-21T07:27:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 -- 172.21.15.105:0/832511225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5bc003a50 msgr2=0x7fd5bc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 --2- 172.21.15.105:0/832511225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5bc003a50 0x7fd5bc003ec0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd5d000c290 tx=0x7fd5bc005830 comp rx=0 tx=0).stop 2024-09-21T07:27:00.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 -- 172.21.15.105:0/832511225 shutdown_connections 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 --2- 172.21.15.105:0/832511225 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5bc003a50 0x7fd5bc003ec0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 -- 172.21.15.105:0/832511225 >> 172.21.15.105:0/832511225 conn(0x7fd5d40cdce0 msgr2=0x7fd5d40ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 -- 172.21.15.105:0/832511225 shutdown_connections 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.446+0000 7fd5daabe640 1 -- 172.21.15.105:0/832511225 wait complete. 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.448+0000 7fd5daabe640 1 Processor -- start 2024-09-21T07:27:00.449 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.448+0000 7fd5daabe640 1 -- start start 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.449+0000 7fd5daabe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.449+0000 7fd5daabe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5d40b1ef0 0x7fd5d40af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5daabe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 0x7fd5d40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5daabe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d415c8b0 con 0x7fd5d40b1ef0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5daabe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d415c730 con 0x7fd5d40b0bc0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5daabe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d415ca30 con 0x7fd5bc003a50 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d905b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 0x7fd5d40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.105:37944/0 (socket says 172.21.15.105:37944) 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 -- 172.21.15.105:0/1854298161 learned_addr learned my addr 172.21.15.105:0/1854298161 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5cbfff640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5d40b1ef0 0x7fd5d40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 -- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 msgr2=0x7fd5d40b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.453 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 0x7fd5d40b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 -- 172.21.15.105:0/1854298161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5d40b1ef0 msgr2=0x7fd5d40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5d40b1ef0 0x7fd5d40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 -- 172.21.15.105:0/1854298161 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d40ae5f0 con 0x7fd5bc003a50 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d905b640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 0x7fd5d40b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.450+0000 7fd5d885a640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002b130 tx=0x7fd5d0004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.451+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0031040 con 0x7fd5bc003a50 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.451+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5d0006420 con 0x7fd5bc003a50 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.451+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d415d450 con 0x7fd5bc003a50 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.451+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0001cd0 con 0x7fd5bc003a50 2024-09-21T07:27:00.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.451+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d415da10 con 0x7fd5bc003a50 2024-09-21T07:27:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.452+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d40b2e10 con 0x7fd5bc003a50 2024-09-21T07:27:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.453+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd5d0034070 con 0x7fd5bc003a50 2024-09-21T07:27:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.453+0000 7fd5c97fa640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 0x7fd5c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.453+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7fd5d00badb0 con 0x7fd5bc003a50 2024-09-21T07:27:00.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.453+0000 7fd5cbfff640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 0x7fd5c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:00.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.454+0000 7fd5cbfff640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 0x7fd5c007b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4154090 tx=0x7fd5c4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:00.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.458+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5d0045060 con 0x7fd5bc003a50 2024-09-21T07:27:00.675 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:00.673+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd5d41e77f0 con 0x7fd5bc003a50 2024-09-21T07:27:01.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.327+0000 7fd5c97fa640 1 -- 172.21.15.105:0/1854298161 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fd5d0082e20 con 0x7fd5bc003a50 2024-09-21T07:27:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 msgr2=0x7fd5c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 0x7fd5c007b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4154090 tx=0x7fd5c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 msgr2=0x7fd5d415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5d002b130 tx=0x7fd5d0004bc0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 shutdown_connections 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fd5c0078b90 0x7fd5c007b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5d40b0bc0 0x7fd5d40b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5d40b1ef0 0x7fd5d40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 --2- 172.21.15.105:0/1854298161 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5bc003a50 0x7fd5d415e210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 >> 172.21.15.105:0/1854298161 conn(0x7fd5d40cdce0 msgr2=0x7fd5d40718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 shutdown_connections 2024-09-21T07:27:01.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.329+0000 7fd5daabe640 1 -- 172.21.15.105:0/1854298161 wait complete. 2024-09-21T07:27:01.343 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:27:01.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 -- 172.21.15.105:0/2629600150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880d0360 msgr2=0x7f83880b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:01.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 --2- 172.21.15.105:0/2629600150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880d0360 0x7f83880b7990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8378009830 tx=0x7f837802a480 comp rx=0 tx=0).stop 2024-09-21T07:27:01.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 -- 172.21.15.105:0/2629600150 shutdown_connections 2024-09-21T07:27:01.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 --2- 172.21.15.105:0/2629600150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 0x7f83880d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 --2- 172.21.15.105:0/2629600150 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880d0360 0x7f83880b7990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 -- 172.21.15.105:0/2629600150 >> 172.21.15.105:0/2629600150 conn(0x7f83880cede0 msgr2=0x7f83880cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 -- 172.21.15.105:0/2629600150 shutdown_connections 2024-09-21T07:27:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.458+0000 7f838e358640 1 -- 172.21.15.105:0/2629600150 wait complete. 2024-09-21T07:27:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.460+0000 7f838e358640 1 Processor -- start 2024-09-21T07:27:01.462 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.460+0000 7f838e358640 1 -- start start 2024-09-21T07:27:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 0x7f838814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 0x7f83881e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f83880d1140 con 0x7f83880b1cc0 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83880d3180 con 0x7f83880d0360 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.461+0000 7f838e358640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f83880d12c0 con 0x7f83880b18b0 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46296/0 (socket says 172.21.15.105:46296) 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 -- 172.21.15.105:0/634840092 learned_addr learned my addr 172.21.15.105:0/634840092 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f8387fff640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 0x7f838814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f838c8f5640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 0x7f83881e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 -- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 msgr2=0x7f838814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 0x7f838814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 -- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 msgr2=0x7f83881e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 0x7f83881e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 -- 172.21.15.105:0/634840092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838814cd50 con 0x7f83880b1cc0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f838c8f5640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 0x7f83881e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f83877fe640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c970 tx=0x7f837c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c007bb0 con 0x7f83880b1cc0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c007d50 con 0x7f83880b1cc0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83881e89e0 con 0x7f83880b1cc0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c005280 con 0x7f83880b1cc0 2024-09-21T07:27:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.462+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83881e8fa0 con 0x7f83880b1cc0 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.463+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83880b7540 con 0x7f83880b1cc0 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.464+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f837c005420 con 0x7f83880b1cc0 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.465+0000 7f8384ff9640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 0x7f837407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.465+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6821+0+0 (secure 0 0 0) 0x7f837c09b550 con 0x7f83880b1cc0 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.465+0000 7f8387fff640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 0x7f837407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.466+0000 7f8387fff640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 0x7f837407b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8378034000 tx=0x7f8378004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.470+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f837c014030 con 0x7f83880b1cc0 2024-09-21T07:27:01.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:01.649+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f83881e8770 con 0x7f83880b1cc0 2024-09-21T07:27:02.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.324+0000 7f8384ff9640 1 -- 172.21.15.105:0/634840092 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f837c062ef0 con 0x7f83880b1cc0 2024-09-21T07:27:02.327 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:27:02.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 msgr2=0x7f837407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 0x7f837407b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8378034000 tx=0x7f8378004730 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 msgr2=0x7f83881e7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f837c00c970 tx=0x7f837c00ce30 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 shutdown_connections 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8374078b90 0x7f837407b0a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83880d0360 0x7f83881e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f83880b1cc0 0x7f83881e7c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 --2- 172.21.15.105:0/634840092 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f83880b18b0 0x7f838814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.328+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 >> 172.21.15.105:0/634840092 conn(0x7f83880cede0 msgr2=0x7f838809aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.329+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 shutdown_connections 2024-09-21T07:27:02.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.329+0000 7f838e358640 1 -- 172.21.15.105:0/634840092 wait complete. 2024-09-21T07:27:02.345 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 -- 172.21.15.105:0/3386702365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 msgr2=0x7f5358168240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 --2- 172.21.15.105:0/3386702365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f5358168240 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5350005fd0 tx=0x7f535002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 -- 172.21.15.105:0/3386702365 shutdown_connections 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 --2- 172.21.15.105:0/3386702365 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f5358168240 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 -- 172.21.15.105:0/3386702365 >> 172.21.15.105:0/3386702365 conn(0x7f53580cede0 msgr2=0x7f53580cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 -- 172.21.15.105:0/3386702365 shutdown_connections 2024-09-21T07:27:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.465+0000 7f535fed3640 1 -- 172.21.15.105:0/3386702365 wait complete. 2024-09-21T07:27:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.467+0000 7f535fed3640 1 Processor -- start 2024-09-21T07:27:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.467+0000 7f535fed3640 1 -- start start 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53580b7990 0x7f535809d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53580d0360 0x7f5358164e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5358075330 con 0x7f5358165e20 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53580d1310 con 0x7f53580d0360 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535fed3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f53580d3180 con 0x7f53580b7990 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46322/0 (socket says 172.21.15.105:46322) 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 -- 172.21.15.105:0/167225348 learned_addr learned my addr 172.21.15.105:0/167225348 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535d46e640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53580d0360 0x7f5358164e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535dc6f640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53580b7990 0x7f535809d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 -- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53580b7990 msgr2=0x7f535809d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53580b7990 0x7f535809d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 -- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53580d0360 msgr2=0x7f5358164e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53580d0360 0x7f5358164e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.468+0000 7f535e470640 1 -- 172.21.15.105:0/167225348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535809da60 con 0x7f5358165e20 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f535e470640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5350009bf0 tx=0x7f5350003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350044070 con 0x7f5358165e20 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5350004250 con 0x7f5358165e20 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535809dd40 con 0x7f5358165e20 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350004660 con 0x7f5358165e20 2024-09-21T07:27:02.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.469+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535815ecd0 con 0x7f5358165e20 2024-09-21T07:27:02.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.470+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5358069b30 con 0x7f5358165e20 2024-09-21T07:27:02.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.471+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5350047070 con 0x7f5358165e20 2024-09-21T07:27:02.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.471+0000 7f534e7fc640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 0x7f533c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:02.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.471+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6383+0+0 (secure 0 0 0) 0x7f53500bbf10 con 0x7f5358165e20 2024-09-21T07:27:02.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.471+0000 7f535dc6f640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 0x7f533c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:02.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.472+0000 7f535dc6f640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 0x7f533c07b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:02.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.476+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5358069b30 con 0x7f5358165e20 2024-09-21T07:27:02.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:02.652+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f535815ef30 con 0x7f5358165e20 2024-09-21T07:27:03.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.330+0000 7f534e7fc640 1 -- 172.21.15.105:0/167225348 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f5350032030 con 0x7f5358165e20 2024-09-21T07:27:03.332 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 msgr2=0x7f533c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 0x7f533c07b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354009040 comp rx=0 tx=0).stop 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 msgr2=0x7f535815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5350009bf0 tx=0x7f5350003d00 comp rx=0 tx=0).stop 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 shutdown_connections 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f533c078b90 0x7f533c07b0a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53580d0360 0x7f5358164e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5358165e20 0x7f535815c140 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 --2- 172.21.15.105:0/167225348 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f53580b7990 0x7f535809d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 >> 172.21.15.105:0/167225348 conn(0x7f53580cede0 msgr2=0x7f535815b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 shutdown_connections 2024-09-21T07:27:03.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.334+0000 7f535fed3640 1 -- 172.21.15.105:0/167225348 wait complete. 2024-09-21T07:27:03.350 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-21T07:27:03.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 -- 172.21.15.105:0/461967033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040cfe50 msgr2=0x7f9d040d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/461967033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040cfe50 0x7f9d040d0260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4005fd0 tx=0x7f9cf402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 -- 172.21.15.105:0/461967033 shutdown_connections 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/461967033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040cfe50 0x7f9d040d0260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/461967033 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040b1b20 0x7f9d040b79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 -- 172.21.15.105:0/461967033 >> 172.21.15.105:0/461967033 conn(0x7f9d040b3e00 msgr2=0x7f9d040b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 -- 172.21.15.105:0/461967033 shutdown_connections 2024-09-21T07:27:03.468 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.465+0000 7f9d0be0a640 1 -- 172.21.15.105:0/461967033 wait complete. 2024-09-21T07:27:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.467+0000 7f9d0be0a640 1 Processor -- start 2024-09-21T07:27:03.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.467+0000 7f9d0be0a640 1 -- start start 2024-09-21T07:27:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.468+0000 7f9d0be0a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d040b1b20 0x7f9d04150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040cfe50 0x7f9d041f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040ceb30 con 0x7f9d040d0a40 2024-09-21T07:27:03.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9d040ce9b0 con 0x7f9d040b1b20 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f9d040b7310 con 0x7f9d040cfe50 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46330/0 (socket says 172.21.15.105:46330) 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 -- 172.21.15.105:0/185686052 learned_addr learned my addr 172.21.15.105:0/185686052 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d09ba6640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d040b1b20 0x7f9d04150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d093a5640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040cfe50 0x7f9d041f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 -- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040cfe50 msgr2=0x7f9d041f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040cfe50 0x7f9d041f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 -- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d040b1b20 msgr2=0x7f9d04150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d040b1b20 0x7f9d04150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 -- 172.21.15.105:0/185686052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d041f7640 con 0x7f9d040d0a40 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0a3a7640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc002230 tx=0x7f9cfc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:03.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc003b70 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cfc003d10 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d041f3630 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc005330 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.469+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d040c6970 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.470+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d040d0f80 con 0x7f9d040d0a40 2024-09-21T07:27:03.473 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.471+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9cfc0056e0 con 0x7f9d040d0a40 2024-09-21T07:27:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.472+0000 7f9cfa7fc640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 0x7f9ce807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.472+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5853+0+0 (secure 0 0 0) 0x7f9cfc09acb0 con 0x7f9d040d0a40 2024-09-21T07:27:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.472+0000 7f9d09ba6640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 0x7f9ce807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:03.474 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.473+0000 7f9d09ba6640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 0x7f9ce807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009b30 tx=0x7f9d00005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:03.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.477+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9cfc062a10 con 0x7f9d040d0a40 2024-09-21T07:27:03.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:03.654+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9d040c9010 con 0x7f9d040d0a40 2024-09-21T07:27:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.339+0000 7f9cfa7fc640 1 -- 172.21.15.105:0/185686052 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f9cfc067440 con 0x7f9d040d0a40 2024-09-21T07:27:04.341 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data_ec' removed 2024-09-21T07:27:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 msgr2=0x7f9ce807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 0x7f9ce807b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009b30 tx=0x7f9d00005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 msgr2=0x7f9d0415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:04.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc002230 tx=0x7f9cfc00cb90 comp rx=0 tx=0).stop 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 shutdown_connections 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f9ce8078b90 0x7f9ce807b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9d040d0a40 0x7f9d0415f990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9d040cfe50 0x7f9d041f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 --2- 172.21.15.105:0/185686052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9d040b1b20 0x7f9d04150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 >> 172.21.15.105:0/185686052 conn(0x7f9d040b3e00 msgr2=0x7f9d040b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 shutdown_connections 2024-09-21T07:27:04.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.342+0000 7f9d0be0a640 1 -- 172.21.15.105:0/185686052 wait complete. 2024-09-21T07:27:04.359 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:27:04.359 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:27:04.360 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:27:04.360 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:27:04.360 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:27:04.360 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:27:04.360 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:27:04.360 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:27:04.360 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:27:04.361 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:27:04.361 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:27:04.361 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:27:04.361 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:27:04.361 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:27:04.361 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:27:04.361 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:27:04.370 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:27:04.370 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:27:04.371 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:27:04.371 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:27:04.372 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:27:04.372 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:27:04.375 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:27:04.375 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:27:04.413 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:27:04.413 DEBUG:teuthology.parallel:result is None 2024-09-21T07:27:04.414 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:27:04.414 DEBUG:teuthology.parallel:result is None 2024-09-21T07:27:04.416 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:27:04.417 DEBUG:teuthology.parallel:result is None 2024-09-21T07:27:04.418 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:27:04.418 DEBUG:teuthology.parallel:result is None 2024-09-21T07:27:04.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:27:04.502 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-09-21T07:27:04.503 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2024-09-21T07:27:04.507 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2024-09-21T07:27:04.508 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2024-09-21T07:27:04.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.539+0000 7f139a12b640 1 -- 172.21.15.105:0/31086070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 msgr2=0x7f137c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.539+0000 7f139a12b640 1 --2- 172.21.15.105:0/31086070 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 0x7f137c003e70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1384009d10 tx=0x7f137c005830 comp rx=0 tx=0).stop 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.539+0000 7f1393fff640 1 -- 172.21.15.105:0/31086070 reap_dead start 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.540+0000 7f139a12b640 1 -- 172.21.15.105:0/31086070 shutdown_connections 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.540+0000 7f139a12b640 1 -- 172.21.15.105:0/31086070 >> 172.21.15.105:0/31086070 conn(0x7f139408bfb0 msgr2=0x7f139409d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.540+0000 7f139a12b640 1 -- 172.21.15.105:0/31086070 shutdown_connections 2024-09-21T07:27:04.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.540+0000 7f139a12b640 1 -- 172.21.15.105:0/31086070 wait complete. 2024-09-21T07:27:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.541+0000 7f139a12b640 1 Processor -- start 2024-09-21T07:27:04.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.542+0000 7f139a12b640 1 -- start start 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 0x7f13941f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13940b7400 0x7f13940b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f13940d0340 con 0x7f13940cb730 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f13940d01c0 con 0x7f137c003a00 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f139a12b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f13940d1ae0 con 0x7f13940b7400 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46376/0 (socket says 172.21.15.105:46376) 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 -- 172.21.15.105:0/343813747 learned_addr learned my addr 172.21.15.105:0/343813747 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f13937fe640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 0x7f13941f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1393fff640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13940b7400 0x7f13940b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:04.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 -- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13940b7400 msgr2=0x7f13940b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13940b7400 0x7f13940b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 -- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 msgr2=0x7f13941f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 0x7f13941f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 -- 172.21.15.105:0/343813747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13940b7d50 con 0x7f13940cb730 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.543+0000 7f1392ffd640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1388011b10 tx=0x7f1388011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.544+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138801e070 con 0x7f13940cb730 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.544+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139414a600 con 0x7f13940cb730 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.544+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1388004460 con 0x7f13940cb730 2024-09-21T07:27:04.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.544+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138800fb80 con 0x7f13940cb730 2024-09-21T07:27:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.544+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139414abc0 con 0x7f13940cb730 2024-09-21T07:27:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.545+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13940d4870 con 0x7f13940cb730 2024-09-21T07:27:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.545+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1388021070 con 0x7f13940cb730 2024-09-21T07:27:04.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.546+0000 7f137bfff640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 0x7f138007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:04.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.546+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7f138809b4b0 con 0x7f13940cb730 2024-09-21T07:27:04.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.546+0000 7f13937fe640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 0x7f138007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:04.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.546+0000 7f13937fe640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 0x7f138007b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1384034000 tx=0x7f13840046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:04.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.550+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13940b7d50 con 0x7f13940cb730 2024-09-21T07:27:04.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:04.746+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f13940b6c90 con 0x7f13940cb730 2024-09-21T07:27:05.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.350+0000 7f137bfff640 1 -- 172.21.15.105:0/343813747 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7f1388063390 con 0x7f13940cb730 2024-09-21T07:27:05.352 INFO:teuthology.orchestra.run.smithi105.stderr: removed all blocklist entries 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 msgr2=0x7f138007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 0x7f138007b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1384034000 tx=0x7f13840046c0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 msgr2=0x7f13941e9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1388011b10 tx=0x7f1388011f10 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 shutdown_connections 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1380078bc0 0x7f138007b0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13940b7400 0x7f13940b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13940cb730 0x7f13941e9be0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 --2- 172.21.15.105:0/343813747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f137c003a00 0x7f13941f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.352+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 >> 172.21.15.105:0/343813747 conn(0x7f139408bfb0 msgr2=0x7f13940b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.353+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 shutdown_connections 2024-09-21T07:27:05.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.353+0000 7f139a12b640 1 -- 172.21.15.105:0/343813747 wait complete. 2024-09-21T07:27:05.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:27:05.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3436712687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0cf1b0 msgr2=0x7fc2ec0cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3436712687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0cf1b0 0x7fc2ec0cf5c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4005fd0 tx=0x7fc2e402a820 comp rx=0 tx=0).stop 2024-09-21T07:27:05.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3436712687 shutdown_connections 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3436712687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0cf1b0 0x7fc2ec0cf5c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3436712687 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0d4460 0x7fc2ec0d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3436712687 >> 172.21.15.105:0/3436712687 conn(0x7fc2ec08bfb0 msgr2=0x7fc2ec09d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3436712687 shutdown_connections 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.480+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3436712687 wait complete. 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.481+0000 7fc2f1a41640 1 Processor -- start 2024-09-21T07:27:05.483 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.482+0000 7fc2f1a41640 1 -- start start 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0cf1b0 0x7fc2ec1ea190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec0d3020 0x7fc2ec1e7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec0cfc80 con 0x7fc2ec0d4460 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ec0cfb00 con 0x7fc2ec0d3020 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2f1a41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ec0cfe00 con 0x7fc2ec0cf1b0 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.486 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46386/0 (socket says 172.21.15.105:46386) 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 -- 172.21.15.105:0/3520018100 learned_addr learned my addr 172.21.15.105:0/3520018100 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2ea7fc640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec0d3020 0x7fc2ec1e7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eaffd640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0cf1b0 0x7fc2ec1ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 -- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0cf1b0 msgr2=0x7fc2ec1ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0cf1b0 0x7fc2ec1ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 -- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec0d3020 msgr2=0x7fc2ec1e7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec0d3020 0x7fc2ec1e7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.483+0000 7fc2eb7fe640 1 -- 172.21.15.105:0/3520018100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ec1f59c0 con 0x7fc2ec0d4460 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2eb7fe640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009b30 tx=0x7fc2e4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4037040 con 0x7fc2ec0d4460 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2e4042070 con 0x7fc2ec0d4460 2024-09-21T07:27:05.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ec1f5ca0 con 0x7fc2ec0d4460 2024-09-21T07:27:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e40042a0 con 0x7fc2ec0d4460 2024-09-21T07:27:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.484+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ec1f6260 con 0x7fc2ec0d4460 2024-09-21T07:27:05.488 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.485+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ec08e500 con 0x7fc2ec0d4460 2024-09-21T07:27:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.485+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2e4004440 con 0x7fc2ec0d4460 2024-09-21T07:27:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.486+0000 7fc2d3fff640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 0x7fc2d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.486+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fc2e40bb780 con 0x7fc2ec0d4460 2024-09-21T07:27:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.486+0000 7fc2eaffd640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 0x7fc2d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.487+0000 7fc2eaffd640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 0x7fc2d807b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c80d0 tx=0x7fc2dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.490+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2ec08e500 con 0x7fc2ec0d4460 2024-09-21T07:27:05.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.735+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc2ec1f5750 con 0x7fc2ec0d4460 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.735+0000 7fc2d3fff640 1 -- 172.21.15.105:0/3520018100 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7fc2e4083730 con 0x7fc2ec0d4460 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "auth_dump": [ 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.737 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.a", 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5mh0dlLRAAaYpF2t+Uk6VjVKs++AFOzw==", 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-09-21T07:27:05.738 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.b", 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5moWd6OhAABsRGlQQJjxGagf8eheSTpw==", 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.739 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.c", 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5m6rIANRAAh2kt5k5Cl1bYkKPJ8K61CA==", 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.740 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mds.d", 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDbdO5mk+KIBhAAEunaRfaOoOnbVUvO+uhnMw==", 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.741 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.0", 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDedO5m8iKKIhAAN3dzLRQ8YMY38vpHsJdxmg==", 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.742 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.1", 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDhdO5m0jA4NBAAu3joL35rLvu39fwQ4MNxkw==", 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.743 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.2", 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDkdO5mxUfnNhAAJ5uQUfTxnxxyDXGntIE0EA==", 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.744 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.3", 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDndO5mXHXIORAAHmBnGC7IXT+ZSSCLGP+eEg==", 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.745 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.4", 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDudO5mQ8KJGhAAMvHlLrf0/OLoR/Wcde5ecA==", 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.746 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.5", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDxdO5mUdbhHBAAXKKvHBseMdFfaGI/5uix2w==", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.6", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD1dO5mW2olARAAHhdnmgsgu9cqwArDMFJIgQ==", 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.747 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "osd.7", 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD4dO5moJCdExAACE4e5trJ+gh0HFXVPpeyvw==", 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow profile osd", 2024-09-21T07:27:05.748 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile osd", 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.0", 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDbdO5mp/ImCxAASyikrNfrfPlhCoeUsnBKSA==", 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.749 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.1", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDbdO5mMYmrEhAAgo1UsOE/3038EBINtmgPaQ==", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-09-21T07:27:05.750 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.2", 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDbdO5mg5TkDhAAe9FnO+jWhlgF35SSGFgvwA==", 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.751 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.3", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDbdO5mxO5lFhAAjiUrrqidE0FkoM5TeXdeuw==", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow r", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow rw", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow rwx" 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.admin", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5mR1uhDhAAX2qp8Nn1XCI0K3VLPUHkNw==", 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.752 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "mgr": "allow *", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow *", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mds", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5m/+isFxAAz43OwhfvQJFp+/NEi2Z1KQ==", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T07:27:05.753 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5mAfisFxAAi98Ur1jiT9LBlqtQWmoYIw==", 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-osd", 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5mPgatFxAALaTXW8A62YF04DgjPDATaQ==", 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.754 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5msROtFxAAi7NtruW3WtOSjEpn8LzlpQ==", 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5mhyWtFxAAg2AvXY/+HbfwZkCGKhTyAg==", 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.755 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQD+dO5mIDOtFxAAjufGzcMKd0q1X7N3j5KaAQ==", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.x", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5m09OYKRAAczmT4tcOHHUPZ4SNYfsyyA==", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-09-21T07:27:05.756 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "entity": "mgr.y", 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "key": "AQDadO5maosdJBAAIyNW23hpwO32rk2iW9Eg5w==", 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": { 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": "allow *", 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "mon": "allow profile mgr", 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: "osd": "allow *" 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-09-21T07:27:05.757 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 msgr2=0x7fc2d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 0x7fc2d807b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0c80d0 tx=0x7fc2dc009040 comp rx=0 tx=0).stop 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 msgr2=0x7fc2ec0ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009b30 tx=0x7fc2e4032c90 comp rx=0 tx=0).stop 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 shutdown_connections 2024-09-21T07:27:05.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc2d8078b90 0x7fc2d807b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2ec0d3020 0x7fc2ec1e7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc2ec0d4460 0x7fc2ec0ca2d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 --2- 172.21.15.105:0/3520018100 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc2ec0cf1b0 0x7fc2ec1ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 >> 172.21.15.105:0/3520018100 conn(0x7fc2ec08bfb0 msgr2=0x7fc2ec0d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 shutdown_connections 2024-09-21T07:27:05.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.737+0000 7fc2f1a41640 1 -- 172.21.15.105:0/3520018100 wait complete. 2024-09-21T07:27:05.760 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T07:27:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 -- 172.21.15.105:0/2465300694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 msgr2=0x7f30600ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.874 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 --2- 172.21.15.105:0/2465300694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f30600ccf40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f30600b74b0 tx=0x7f305802aa80 comp rx=0 tx=0).stop 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 -- 172.21.15.105:0/2465300694 shutdown_connections 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 --2- 172.21.15.105:0/2465300694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f30600ccf40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 --2- 172.21.15.105:0/2465300694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 0x7f30600b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 -- 172.21.15.105:0/2465300694 >> 172.21.15.105:0/2465300694 conn(0x7f30600cedb0 msgr2=0x7f30600cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 -- 172.21.15.105:0/2465300694 shutdown_connections 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.872+0000 7f30650bd640 1 -- 172.21.15.105:0/2465300694 wait complete. 2024-09-21T07:27:05.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.874+0000 7f30650bd640 1 Processor -- start 2024-09-21T07:27:05.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.874+0000 7f30650bd640 1 -- start start 2024-09-21T07:27:05.878 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 0x7f306015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30600cff00 0x7f30601f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f30600b1d10 con 0x7f30600d0310 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30600cfce0 con 0x7f30600cff00 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f30650bd640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f30600b1e90 con 0x7f30600b7940 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f305f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f305f59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46408/0 (socket says 172.21.15.105:46408) 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.875+0000 7f305f59e640 1 -- 172.21.15.105:0/4146131763 learned_addr learned my addr 172.21.15.105:0/4146131763 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305ed9d640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 0x7f306015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305659c640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30600cff00 0x7f30601f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 -- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 msgr2=0x7f306015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 0x7f306015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.879 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 -- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30600cff00 msgr2=0x7f30601f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30600cff00 0x7f30601f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 -- 172.21.15.105:0/4146131763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601f6790 con 0x7f30600d0310 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f305f59e640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3058009d60 tx=0x7f3058003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058044070 con 0x7f30600d0310 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3058003e50 con 0x7f30600d0310 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30600c6520 con 0x7f30600d0310 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004490 con 0x7f30600d0310 2024-09-21T07:27:05.880 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.876+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30600c6ae0 con 0x7f30600d0310 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.877+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30600b17d0 con 0x7f30600d0310 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.878+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3058047070 con 0x7f30600d0310 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.878+0000 7f3057fff640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 0x7f304007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.879+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f30580bb910 con 0x7f30600d0310 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.879+0000 7f305ed9d640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 0x7f304007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:05.881 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.879+0000 7f305ed9d640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 0x7f304007b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3050009b30 tx=0x7f3050005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:05.884 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:05.883+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3058032030 con 0x7f30600d0310 2024-09-21T07:27:06.131 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.129+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f30600b3bb0 con 0x7f30600d0310 2024-09-21T07:27:06.133 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.132+0000 7f3057fff640 1 -- 172.21.15.105:0/4146131763 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f3058083950 con 0x7f30600d0310 2024-09-21T07:27:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 msgr2=0x7f304007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 0x7f304007b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3050009b30 tx=0x7f3050005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:06.136 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 msgr2=0x7f3060159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3058009d60 tx=0x7f3058003ce0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 shutdown_connections 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3040078bc0 0x7f304007b0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30600cff00 0x7f30601f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30600d0310 0x7f3060159180 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 --2- 172.21.15.105:0/4146131763 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30600b7940 0x7f306015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 >> 172.21.15.105:0/4146131763 conn(0x7f30600cedb0 msgr2=0x7f30600b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 shutdown_connections 2024-09-21T07:27:06.137 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.134+0000 7f30650bd640 1 -- 172.21.15.105:0/4146131763 wait complete. 2024-09-21T07:27:06.149 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 -- 172.21.15.105:0/3833530533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 msgr2=0x7f59740b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 --2- 172.21.15.105:0/3833530533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740b3cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f597408d170 tx=0x7f596402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 -- 172.21.15.105:0/3833530533 shutdown_connections 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 --2- 172.21.15.105:0/3833530533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f59740d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 --2- 172.21.15.105:0/3833530533 >> v2:172.21.15.112:3300/0 conn(0x7f59740d4460 0x7f59740d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 --2- 172.21.15.105:0/3833530533 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740b3cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 -- 172.21.15.105:0/3833530533 >> 172.21.15.105:0/3833530533 conn(0x7f597408bfb0 msgr2=0x7f597409d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 -- 172.21.15.105:0/3833530533 shutdown_connections 2024-09-21T07:27:06.263 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.261+0000 7f59797ad640 1 -- 172.21.15.105:0/3833530533 wait complete. 2024-09-21T07:27:06.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.263+0000 7f59797ad640 1 Processor -- start 2024-09-21T07:27:06.264 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.263+0000 7f59797ad640 1 -- start start 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f597414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 0x7f59740d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f59740b4aa0 con 0x7f59740d3020 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59740b4920 con 0x7f59740d4460 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59797ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f59740d1910 con 0x7f59740d0fc0 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46434/0 (socket says 172.21.15.105:46434) 2024-09-21T07:27:06.267 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 -- 172.21.15.105:0/3739136254 learned_addr learned my addr 172.21.15.105:0/3739136254 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f5972ffd640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f597414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59737fe640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 0x7f59740d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 -- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 msgr2=0x7f597414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f597414de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 -- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 msgr2=0x7f59740d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 0x7f59740d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 -- 172.21.15.105:0/3739136254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5974160650 con 0x7f59740d3020 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f5972ffd640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f597414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59737fe640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 0x7f59740d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.264+0000 7f59727fc640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f596800c9a0 tx=0x7f596800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.265+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968007bb0 con 0x7f59740d3020 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.265+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5968007e40 con 0x7f59740d3020 2024-09-21T07:27:06.268 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.265+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597414e370 con 0x7f59740d3020 2024-09-21T07:27:06.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.265+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968005280 con 0x7f59740d3020 2024-09-21T07:27:06.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.265+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741f9950 con 0x7f59740d3020 2024-09-21T07:27:06.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.266+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59740c97b0 con 0x7f59740d3020 2024-09-21T07:27:06.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.267+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5968005420 con 0x7f59740d3020 2024-09-21T07:27:06.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.267+0000 7f595b7fe640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 0x7f596007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.267+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f596809af20 con 0x7f59740d3020 2024-09-21T07:27:06.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.267+0000 7f5972ffd640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 0x7f596007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.268+0000 7f5972ffd640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 0x7f596007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5964034000 tx=0x7f5964004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:06.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.272+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f597414e370 con 0x7f59740d3020 2024-09-21T07:27:06.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.521+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f59740b3cb0 con 0x7f59740d3020 2024-09-21T07:27:06.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.526+0000 7f595b7fe640 1 -- 172.21.15.105:0/3739136254 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f5968014030 con 0x7f59740d3020 2024-09-21T07:27:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 msgr2=0x7f596007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 0x7f596007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5964034000 tx=0x7f5964004730 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 msgr2=0x7f59740d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f596800c9a0 tx=0x7f596800ce60 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 shutdown_connections 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5960078b90 0x7f596007b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59740d4460 0x7f59740d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f59740d3020 0x7f59740d0190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 --2- 172.21.15.105:0/3739136254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f59740d0fc0 0x7f597414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 >> 172.21.15.105:0/3739136254 conn(0x7f597408bfb0 msgr2=0x7f597406f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.529+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 shutdown_connections 2024-09-21T07:27:06.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.530+0000 7f59797ad640 1 -- 172.21.15.105:0/3739136254 wait complete. 2024-09-21T07:27:06.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T07:27:06.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 -- 172.21.15.105:0/516611723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cd090 msgr2=0x7f96c40cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 --2- 172.21.15.105:0/516611723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cd090 0x7f96c40cd4a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f96c4074ee0 tx=0x7f96b802a560 comp rx=0 tx=0).stop 2024-09-21T07:27:06.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 -- 172.21.15.105:0/516611723 shutdown_connections 2024-09-21T07:27:06.656 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 --2- 172.21.15.105:0/516611723 >> v2:172.21.15.112:3300/0 conn(0x7f96c40cd9e0 0x7f96c40b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 --2- 172.21.15.105:0/516611723 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cd090 0x7f96c40cd4a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 --2- 172.21.15.105:0/516611723 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cff70 0x7f96c40d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 -- 172.21.15.105:0/516611723 >> 172.21.15.105:0/516611723 conn(0x7f96c40cee20 msgr2=0x7f96c40cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:06.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 -- 172.21.15.105:0/516611723 shutdown_connections 2024-09-21T07:27:06.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.654+0000 7f96c8acc640 1 -- 172.21.15.105:0/516611723 wait complete. 2024-09-21T07:27:06.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.656+0000 7f96c8acc640 1 Processor -- start 2024-09-21T07:27:06.658 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.656+0000 7f96c8acc640 1 -- start start 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cd090 0x7f96c41e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c40cd9e0 0x7f96c40b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40b7950 con 0x7f96c40cff70 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f96c40b77d0 con 0x7f96c40cd9e0 2024-09-21T07:27:06.662 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.657+0000 7f96c8acc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f96c40cddf0 con 0x7f96c40cd090 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46454/0 (socket says 172.21.15.105:46454) 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 -- 172.21.15.105:0/1875857730 learned_addr learned my addr 172.21.15.105:0/1875857730 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c259c640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cd090 0x7f96c41e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c1d9b640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c40cd9e0 0x7f96c40b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 -- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cd090 msgr2=0x7f96c41e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cd090 0x7f96c41e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 -- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c40cd9e0 msgr2=0x7f96c40b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c40cd9e0 0x7f96c40b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 -- 172.21.15.105:0/1875857730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c41e88d0 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c2d9d640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00ca00 tx=0x7f96bc00cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007cd0 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96bc003990 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c41e7830 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc003de0 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.658+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c41e7df0 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.659+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c41e75a0 con 0x7f96c40cff70 2024-09-21T07:27:06.664 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.660+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96bc004030 con 0x7f96c40cff70 2024-09-21T07:27:06.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.660+0000 7f96b2ffd640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 0x7f96a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:06.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.661+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f96bc09b0d0 con 0x7f96c40cff70 2024-09-21T07:27:06.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.661+0000 7f96c259c640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 0x7f96a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:06.665 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.661+0000 7f96c259c640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 0x7f96a407b0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f96b40058c0 tx=0x7f96b4005f60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:06.667 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.665+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96bc014070 con 0x7f96c40cff70 2024-09-21T07:27:06.914 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.913+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f96c40cd4a0 con 0x7f96c40cff70 2024-09-21T07:27:06.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.918+0000 7f96b2ffd640 1 -- 172.21.15.105:0/1875857730 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f96bc063110 con 0x7f96c40cff70 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.919+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 msgr2=0x7f96a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.919+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 0x7f96a407b0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f96b40058c0 tx=0x7f96b4005f60 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.919+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 msgr2=0x7f96c40b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.919+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00ca00 tx=0x7f96bc00cec0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.919+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 shutdown_connections 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f96a4078bc0 0x7f96a407b0d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f96c40cd9e0 0x7f96c40b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f96c40cff70 0x7f96c40b2a50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 --2- 172.21.15.105:0/1875857730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f96c40cd090 0x7f96c41e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 >> 172.21.15.105:0/1875857730 conn(0x7f96c40cee20 msgr2=0x7f96c41e5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:06.922 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 shutdown_connections 2024-09-21T07:27:06.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:06.920+0000 7f96c8acc640 1 -- 172.21.15.105:0/1875857730 wait complete. 2024-09-21T07:27:06.933 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 -- 172.21.15.105:0/701771249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 msgr2=0x7f69980d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 --2- 172.21.15.105:0/701771249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69980d4870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f699808d450 tx=0x7f698002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 -- 172.21.15.105:0/701771249 shutdown_connections 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 --2- 172.21.15.105:0/701771249 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69980d4870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 --2- 172.21.15.105:0/701771249 >> v2:172.21.15.112:3300/0 conn(0x7f69980d3020 0x7f69980b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 -- 172.21.15.105:0/701771249 >> 172.21.15.105:0/701771249 conn(0x7f699808bfb0 msgr2=0x7f699809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 -- 172.21.15.105:0/701771249 shutdown_connections 2024-09-21T07:27:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.044+0000 7f699d8cc640 1 -- 172.21.15.105:0/701771249 wait complete. 2024-09-21T07:27:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.045+0000 7f699d8cc640 1 Processor -- start 2024-09-21T07:27:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.046+0000 7f699d8cc640 1 -- start start 2024-09-21T07:27:07.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 0x7f69981f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69980d3020 0x7f6998153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f69980b4aa0 con 0x7f69980d4460 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69980b4920 con 0x7f69980d0fc0 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f699d8cc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f69980d01c0 con 0x7f69980d3020 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46472/0 (socket says 172.21.15.105:46472) 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 -- 172.21.15.105:0/4144276820 learned_addr learned my addr 172.21.15.105:0/4144276820 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f6996ffd640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 0x7f69981f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69967fc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69980d3020 0x7f6998153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 -- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69980d3020 msgr2=0x7f6998153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69980d3020 0x7f6998153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 -- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 msgr2=0x7f69981f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 0x7f69981f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.051 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f69977fe640 1 -- 172.21.15.105:0/4144276820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6998153920 con 0x7f69980d4460 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.047+0000 7f6996ffd640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 0x7f69981f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f69977fe640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f699000c3c0 tx=0x7f699000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990017070 con 0x7f69980d4460 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69900035f0 con 0x7f69980d4460 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69981f96e0 con 0x7f69980d4460 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990003a40 con 0x7f69980d4460 2024-09-21T07:27:07.052 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.048+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981f5620 con 0x7f69980d4460 2024-09-21T07:27:07.053 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.049+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69981f9450 con 0x7f69980d4460 2024-09-21T07:27:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.049+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f699001a070 con 0x7f69980d4460 2024-09-21T07:27:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.050+0000 7f698ffff640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 0x7f697807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.050+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f6990099cb0 con 0x7f69980d4460 2024-09-21T07:27:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.050+0000 7f6996ffd640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 0x7f697807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.054 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.051+0000 7f6996ffd640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 0x7f697807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6988008d40 tx=0x7f69880082b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.056 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.054+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6990062400 con 0x7f69980d4460 2024-09-21T07:27:07.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.301+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f69980b7e40 con 0x7f69980d4460 2024-09-21T07:27:07.307 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.306+0000 7f698ffff640 1 -- 172.21.15.105:0/4144276820 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f6990061d00 con 0x7f69980d4460 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 msgr2=0x7f697807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 0x7f697807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6988008d40 tx=0x7f69880082b0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 msgr2=0x7f69981f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f699000c3c0 tx=0x7f699000c880 comp rx=0 tx=0).stop 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 shutdown_connections 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6978078b90 0x7f697807b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f69980d4460 0x7f69981f8b00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f69980d3020 0x7f6998153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 --2- 172.21.15.105:0/4144276820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69980d0fc0 0x7f69981f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 >> 172.21.15.105:0/4144276820 conn(0x7f699808bfb0 msgr2=0x7f69980b8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 shutdown_connections 2024-09-21T07:27:07.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.309+0000 7f699d8cc640 1 -- 172.21.15.105:0/4144276820 wait complete. 2024-09-21T07:27:07.322 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T07:27:07.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 -- 172.21.15.105:0/446172910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0b1b00 msgr2=0x7f5dec0b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 --2- 172.21.15.105:0/446172910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0b1b00 0x7f5dec0b1f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009860 tx=0x7f5ddc02a510 comp rx=0 tx=0).stop 2024-09-21T07:27:07.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 -- 172.21.15.105:0/446172910 shutdown_connections 2024-09-21T07:27:07.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 --2- 172.21.15.105:0/446172910 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0b1b00 0x7f5dec0b1f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 -- 172.21.15.105:0/446172910 >> 172.21.15.105:0/446172910 conn(0x7f5dec0cee20 msgr2=0x7f5dec0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 -- 172.21.15.105:0/446172910 shutdown_connections 2024-09-21T07:27:07.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.434+0000 7f5df3d94640 1 -- 172.21.15.105:0/446172910 wait complete. 2024-09-21T07:27:07.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.436+0000 7f5df3d94640 1 Processor -- start 2024-09-21T07:27:07.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.436+0000 7f5df3d94640 1 -- start start 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 0x7f5dec0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 0x7f5dec0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec09bbb0 con 0x7f5dec0b1b00 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec09ba30 con 0x7f5dec0b2450 2024-09-21T07:27:07.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df3d94640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec0b2860 con 0x7f5dec0cff70 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df1b30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df1b30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46478/0 (socket says 172.21.15.105:46478) 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.437+0000 7f5df1b30640 1 -- 172.21.15.105:0/3443090465 learned_addr learned my addr 172.21.15.105:0/3443090465 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df132f640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 0x7f5dec0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df2331640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 0x7f5dec0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 -- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 msgr2=0x7f5dec0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 0x7f5dec0b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 -- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 msgr2=0x7f5dec0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 0x7f5dec0b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 -- 172.21.15.105:0/3443090465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1f70d0 con 0x7f5dec0b1b00 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df2331640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 0x7f5dec0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df132f640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 0x7f5dec0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df1b30640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c440 tx=0x7f5de800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8016020 con 0x7f5dec0b1b00 2024-09-21T07:27:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de8012920 con 0x7f5dec0b1b00 2024-09-21T07:27:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8012d70 con 0x7f5dec0b1b00 2024-09-21T07:27:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec1f73b0 con 0x7f5dec0b1b00 2024-09-21T07:27:07.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.438+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1f7970 con 0x7f5dec0b1b00 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.439+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec072fe0 con 0x7f5dec0b1b00 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.440+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5de8019020 con 0x7f5dec0b1b00 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.440+0000 7f5de27fc640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 0x7f5dd007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.440+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5de809a570 con 0x7f5dec0b1b00 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.441+0000 7f5df132f640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 0x7f5dd007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.441+0000 7f5df132f640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 0x7f5dd007b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc034000 tx=0x7f5ddc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.445+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5dec072fe0 con 0x7f5dec0b1b00 2024-09-21T07:27:07.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.691+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f5dec0b1f10 con 0x7f5dec0b1b00 2024-09-21T07:27:07.697 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.696+0000 7f5de27fc640 1 -- 172.21.15.105:0/3443090465 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f5de8020070 con 0x7f5dec0b1b00 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 msgr2=0x7f5dd007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 0x7f5dd007b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc034000 tx=0x7f5ddc004730 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 msgr2=0x7f5dec1e5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c440 tx=0x7f5de800c900 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 shutdown_connections 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5dd0078bc0 0x7f5dd007b0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5dec0b2450 0x7f5dec0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5dec0cff70 0x7f5dec0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 --2- 172.21.15.105:0/3443090465 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5dec0b1b00 0x7f5dec1e5ff0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 >> 172.21.15.105:0/3443090465 conn(0x7f5dec0cee20 msgr2=0x7f5dec072820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 shutdown_connections 2024-09-21T07:27:07.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.698+0000 7f5df3d94640 1 -- 172.21.15.105:0/3443090465 wait complete. 2024-09-21T07:27:07.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.822+0000 7f8a00cbe640 1 -- 172.21.15.105:0/3951461113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc0cc1d0 msgr2=0x7f89fc073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.822+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/3951461113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc0cc1d0 0x7f89fc073dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005fd0 tx=0x7f89e402a540 comp rx=0 tx=0).stop 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 -- 172.21.15.105:0/3951461113 shutdown_connections 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/3951461113 >> v2:172.21.15.112:3300/0 conn(0x7f89fc0ca760 0x7f89fc0cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/3951461113 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc0cc1d0 0x7f89fc073dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 -- 172.21.15.105:0/3951461113 >> 172.21.15.105:0/3951461113 conn(0x7f89fc0ccb90 msgr2=0x7f89fc0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 -- 172.21.15.105:0/3951461113 shutdown_connections 2024-09-21T07:27:07.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.823+0000 7f8a00cbe640 1 -- 172.21.15.105:0/3951461113 wait complete. 2024-09-21T07:27:07.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.824+0000 7f8a00cbe640 1 Processor -- start 2024-09-21T07:27:07.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.825+0000 7f8a00cbe640 1 -- start start 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc0cb4a0 0x7f89fc0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc1e7ce0 0x7f89fc0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc0ceb50 con 0x7f89fc0ca760 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89fc0ce9d0 con 0x7f89fc0cb4a0 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f8a00cbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f89fc0cecd0 con 0x7f89fc1e7ce0 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46484/0 (socket says 172.21.15.105:46484) 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 -- 172.21.15.105:0/2973549799 learned_addr learned my addr 172.21.15.105:0/2973549799 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fad9d640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc1e7ce0 0x7f89fc0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89f9d9b640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc0cb4a0 0x7f89fc0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 -- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc1e7ce0 msgr2=0x7f89fc0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc1e7ce0 0x7f89fc0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 -- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc0cb4a0 msgr2=0x7f89fc0c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc0cb4a0 0x7f89fc0c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 -- 172.21.15.105:0/2973549799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89fc0c8b10 con 0x7f89fc0ca760 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.826+0000 7f89fa59c640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c760 tx=0x7f89ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.827+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec003b70 con 0x7f89fc0ca760 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.827+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89ec003d30 con 0x7f89fc0ca760 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.827+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89fc0c8df0 con 0x7f89fc0ca760 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.827+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00aab0 con 0x7f89fc0ca760 2024-09-21T07:27:07.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.827+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89fc0c93b0 con 0x7f89fc0ca760 2024-09-21T07:27:07.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.828+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89fc0cdf20 con 0x7f89fc0ca760 2024-09-21T07:27:07.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.828+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89ec00ac50 con 0x7f89fc0ca760 2024-09-21T07:27:07.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.829+0000 7f89f2ffd640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 0x7f89dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:07.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.829+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f89ec09aa60 con 0x7f89fc0ca760 2024-09-21T07:27:07.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.829+0000 7f89f9d9b640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 0x7f89dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:07.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.830+0000 7f89f9d9b640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 0x7f89dc07b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89e4033000 tx=0x7f89e4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:07.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:07.834+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89ec062fe0 con 0x7f89fc0ca760 2024-09-21T07:27:08.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.078+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f89fc0ad780 con 0x7f89fc0ca760 2024-09-21T07:27:08.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.083+0000 7f89f2ffd640 1 -- 172.21.15.105:0/2973549799 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f89ec0628e0 con 0x7f89fc0ca760 2024-09-21T07:27:08.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 msgr2=0x7f89dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 0x7f89dc07b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89e4033000 tx=0x7f89e4004730 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 msgr2=0x7f89fc14e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c760 tx=0x7f89ec00cc20 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 shutdown_connections 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f89dc078b90 0x7f89dc07b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89fc0cb4a0 0x7f89fc0c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89fc1e7ce0 0x7f89fc0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 --2- 172.21.15.105:0/2973549799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89fc0ca760 0x7f89fc14e420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.085+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 >> 172.21.15.105:0/2973549799 conn(0x7f89fc0ccb90 msgr2=0x7f89fc0adca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.086+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 shutdown_connections 2024-09-21T07:27:08.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.086+0000 7f8a00cbe640 1 -- 172.21.15.105:0/2973549799 wait complete. 2024-09-21T07:27:08.099 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 -- 172.21.15.105:0/1619533424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c980b6850 msgr2=0x7f1c980cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/1619533424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c980b6850 0x7f1c980cf590 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c9806ff60 tx=0x7f1c9002aa00 comp rx=0 tx=0).stop 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 -- 172.21.15.105:0/1619533424 shutdown_connections 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/1619533424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c980b6850 0x7f1c980cf590 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/1619533424 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 0x7f1c980cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/1619533424 >> v2:172.21.15.112:3300/0 conn(0x7f1c9808c3c0 0x7f1c9806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 -- 172.21.15.105:0/1619533424 >> 172.21.15.105:0/1619533424 conn(0x7f1c9808bfb0 msgr2=0x7f1c980d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 -- 172.21.15.105:0/1619533424 shutdown_connections 2024-09-21T07:27:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.209+0000 7f1c9dbda640 1 -- 172.21.15.105:0/1619533424 wait complete. 2024-09-21T07:27:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.211+0000 7f1c9dbda640 1 Processor -- start 2024-09-21T07:27:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.211+0000 7f1c9dbda640 1 -- start start 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 0x7f1c980b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c980b6850 0x7f1c980b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980cfad0 con 0x7f1c9808c3c0 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980b3b60 con 0x7f1c980b6850 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c9dbda640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1c980cfc50 con 0x7f1c980b6440 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46508/0 (socket says 172.21.15.105:46508) 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 -- 172.21.15.105:0/2439401224 learned_addr learned my addr 172.21.15.105:0/2439401224 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:08.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c96ffd640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 0x7f1c980b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c97fff640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c980b6850 0x7f1c980b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 -- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 msgr2=0x7f1c980b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 0x7f1c980b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 -- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c980b6850 msgr2=0x7f1c980b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c980b6850 0x7f1c980b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.212+0000 7f1c977fe640 1 -- 172.21.15.105:0/2439401224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9815f830 con 0x7f1c9808c3c0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c977fe640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800d650 tx=0x7f1c8800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88017070 con 0x7f1c9808c3c0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c88004310 con 0x7f1c9808c3c0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9815fb10 con 0x7f1c9808c3c0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88005280 con 0x7f1c9808c3c0 2024-09-21T07:27:08.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.213+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c981600d0 con 0x7f1c9808c3c0 2024-09-21T07:27:08.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.214+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c9806f7f0 con 0x7f1c9808c3c0 2024-09-21T07:27:08.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.215+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c88005420 con 0x7f1c9808c3c0 2024-09-21T07:27:08.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.215+0000 7f1c7ffff640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 0x7f1c8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.215+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f1c8809ae80 con 0x7f1c9808c3c0 2024-09-21T07:27:08.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.216+0000 7f1c96ffd640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 0x7f1c8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.216+0000 7f1c96ffd640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 0x7f1c8407b0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c980c7f10 tx=0x7f1c8c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:08.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.220+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c88020020 con 0x7f1c9808c3c0 2024-09-21T07:27:08.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.398+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1c980b6d80 con 0x7f1c9808c3c0 2024-09-21T07:27:08.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.398+0000 7f1c7ffff640 1 -- 172.21.15.105:0/2439401224 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7f1c88063510 con 0x7f1c9808c3c0 2024-09-21T07:27:08.400 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:08.401 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 msgr2=0x7f1c8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 0x7f1c8407b0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1c980c7f10 tx=0x7f1c8c005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 msgr2=0x7f1c98157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1c8800d650 tx=0x7f1c8800db10 comp rx=0 tx=0).stop 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 shutdown_connections 2024-09-21T07:27:08.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f1c84078bc0 0x7f1c8407b0d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c980b6850 0x7f1c980b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c980b6440 0x7f1c980b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 --2- 172.21.15.105:0/2439401224 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c9808c3c0 0x7f1c98157df0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.400+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 >> 172.21.15.105:0/2439401224 conn(0x7f1c9808bfb0 msgr2=0x7f1c980d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.401+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 shutdown_connections 2024-09-21T07:27:08.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.401+0000 7f1c9dbda640 1 -- 172.21.15.105:0/2439401224 wait complete. 2024-09-21T07:27:08.414 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:27:08.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:27:08.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 -- 172.21.15.105:0/211928120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f20004d90 msgr2=0x7f5f20005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/211928120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f20004d90 0x7f5f20005260 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c640 tx=0x7f5f20005830 comp rx=0 tx=0).stop 2024-09-21T07:27:08.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f37fff640 1 -- 172.21.15.105:0/211928120 reap_dead start 2024-09-21T07:27:08.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 -- 172.21.15.105:0/211928120 shutdown_connections 2024-09-21T07:27:08.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 -- 172.21.15.105:0/211928120 >> 172.21.15.105:0/211928120 conn(0x7f5f380d4530 msgr2=0x7f5f380d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:08.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 -- 172.21.15.105:0/211928120 shutdown_connections 2024-09-21T07:27:08.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.530+0000 7f5f3dcde640 1 -- 172.21.15.105:0/211928120 wait complete. 2024-09-21T07:27:08.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.532+0000 7f5f3dcde640 1 Processor -- start 2024-09-21T07:27:08.534 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.532+0000 7f5f3dcde640 1 -- start start 2024-09-21T07:27:08.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f20004d90 0x7f5f381510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f381544a0 0x7f5f380ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b9df0 con 0x7f5f38153b50 2024-09-21T07:27:08.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f380b9c70 con 0x7f5f381544a0 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5f380d32e0 con 0x7f5f20004d90 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46534/0 (socket says 172.21.15.105:46534) 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 -- 172.21.15.105:0/2422300791 learned_addr learned my addr 172.21.15.105:0/2422300791 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f20004d90 0x7f5f381510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f37fff640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f381544a0 0x7f5f380ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 -- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f20004d90 msgr2=0x7f5f381510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f20004d90 0x7f5f381510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 -- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f381544a0 msgr2=0x7f5f380ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f381544a0 0x7f5f380ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 -- 172.21.15.105:0/2422300791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f380cac80 con 0x7f5f38153b50 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f36ffd640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00b5b0 tx=0x7f5f2c00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c005220 con 0x7f5f38153b50 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f2c0053c0 con 0x7f5f38153b50 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2c0047a0 con 0x7f5f38153b50 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f380caf60 con 0x7f5f38153b50 2024-09-21T07:27:08.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.534+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f380cdc00 con 0x7f5f38153b50 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.535+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f381f61c0 con 0x7f5f38153b50 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.536+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f2c004940 con 0x7f5f38153b50 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.536+0000 7f5f1ffff640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 0x7f5f2407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.537+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5f2c09b760 con 0x7f5f38153b50 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.537+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 0x7f5f2407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:08.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.537+0000 7f5f377fe640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 0x7f5f2407b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f28003640 tx=0x7f5f2800b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:08.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.541+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f2c063ce0 con 0x7f5f38153b50 2024-09-21T07:27:08.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:08.722+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5f3806f050 con 0x7f5f38153b50 2024-09-21T07:27:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.369+0000 7f5f1ffff640 1 -- 172.21.15.105:0/2422300791 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7f5f2c0635e0 con 0x7f5f38153b50 2024-09-21T07:27:09.371 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 msgr2=0x7f5f2407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 0x7f5f2407b0d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f28003640 tx=0x7f5f2800b260 comp rx=0 tx=0).stop 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 msgr2=0x7f5f38153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c00b5b0 tx=0x7f5f2c00ba70 comp rx=0 tx=0).stop 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 shutdown_connections 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5f24078bc0 0x7f5f2407b0d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f381544a0 0x7f5f380ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f38153b50 0x7f5f38153f60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 --2- 172.21.15.105:0/2422300791 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f20004d90 0x7f5f381510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 >> 172.21.15.105:0/2422300791 conn(0x7f5f380d4530 msgr2=0x7f5f380b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:09.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 shutdown_connections 2024-09-21T07:27:09.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.372+0000 7f5f3dcde640 1 -- 172.21.15.105:0/2422300791 wait complete. 2024-09-21T07:27:09.391 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:27:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 -- 172.21.15.105:0/1249052213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d4004800 msgr2=0x7f11d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 --2- 172.21.15.105:0/1249052213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d4004800 0x7f11d4004c10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009b50 tx=0x7f11d4005830 comp rx=0 tx=0).stop 2024-09-21T07:27:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 -- 172.21.15.105:0/1249052213 shutdown_connections 2024-09-21T07:27:09.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 --2- 172.21.15.105:0/1249052213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11d4004800 0x7f11d4004c10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 -- 172.21.15.105:0/1249052213 >> 172.21.15.105:0/1249052213 conn(0x7f11ec0d1270 msgr2=0x7f11ec0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 -- 172.21.15.105:0/1249052213 shutdown_connections 2024-09-21T07:27:09.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.511+0000 7f11f1038640 1 -- 172.21.15.105:0/1249052213 wait complete. 2024-09-21T07:27:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.513+0000 7f11f1038640 1 Processor -- start 2024-09-21T07:27:09.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.513+0000 7f11f1038640 1 -- start start 2024-09-21T07:27:09.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d4004800 0x7f11ec1df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11ec0af310 0x7f11ec0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec0b0ae0 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec0b69a0 con 0x7f11d4004800 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.514+0000 7f11f1038640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f11ec0b0c60 con 0x7f11ec0af310 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11ea59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11ec0af310 0x7f11ec0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11ead9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d4004800 0x7f11ec1df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46546/0 (socket says 172.21.15.105:46546) 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 -- 172.21.15.105:0/3286993728 learned_addr learned my addr 172.21.15.105:0/3286993728 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 -- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11ec0af310 msgr2=0x7f11ec0af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11ec0af310 0x7f11ec0af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 -- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d4004800 msgr2=0x7f11ec1df1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d4004800 0x7f11ec1df1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 -- 172.21.15.105:0/3286993728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec1df6f0 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11eb59e640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f11e4001330 tx=0x7f11e40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4011870 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1f7970 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11e4017070 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e4011dc0 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.515+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec0ae830 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.517+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec1f76e0 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.517+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11e401c460 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.518+0000 7f11d37fe640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 0x7f11d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.518+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7f11e409b330 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.518+0000 7f11ead9d640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 0x7f11d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:09.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.518+0000 7f11ead9d640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 0x7f11d807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11dc001220 tx=0x7f11dc006720 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:09.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.522+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11e4063180 con 0x7f11ec1cb0f0 2024-09-21T07:27:09.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:09.699+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f11ec0b40c0 con 0x7f11ec1cb0f0 2024-09-21T07:27:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.371+0000 7f11d37fe640 1 -- 172.21.15.105:0/3286993728 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7f11e406a0d0 con 0x7f11ec1cb0f0 2024-09-21T07:27:10.373 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2024-09-21T07:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 msgr2=0x7f11d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 0x7f11d807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11dc001220 tx=0x7f11dc006720 comp rx=0 tx=0).stop 2024-09-21T07:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 msgr2=0x7f11ec1cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f11e4001330 tx=0x7f11e40019b0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 shutdown_connections 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f11d8078bc0 0x7f11d807b0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f11ec1cb0f0 0x7f11ec1cb500 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f11ec0af310 0x7f11ec0af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 --2- 172.21.15.105:0/3286993728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f11d4004800 0x7f11ec1df1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 >> 172.21.15.105:0/3286993728 conn(0x7f11ec0d1270 msgr2=0x7f11ec071910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 shutdown_connections 2024-09-21T07:27:10.377 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.374+0000 7f11f1038640 1 -- 172.21.15.105:0/3286993728 wait complete. 2024-09-21T07:27:10.401 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 -- 172.21.15.105:0/3123026121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 msgr2=0x7f10c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 --2- 172.21.15.105:0/3123026121 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10c4003e70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f10c8005fd0 tx=0x7f10c4005830 comp rx=0 tx=0).stop 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10d7fff640 1 -- 172.21.15.105:0/3123026121 reap_dead start 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 -- 172.21.15.105:0/3123026121 shutdown_connections 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 -- 172.21.15.105:0/3123026121 >> 172.21.15.105:0/3123026121 conn(0x7f10d80cdce0 msgr2=0x7f10d80ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 -- 172.21.15.105:0/3123026121 shutdown_connections 2024-09-21T07:27:10.521 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.519+0000 7f10de0a9640 1 -- 172.21.15.105:0/3123026121 wait complete. 2024-09-21T07:27:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.520+0000 7f10de0a9640 1 Processor -- start 2024-09-21T07:27:10.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.521+0000 7f10de0a9640 1 -- start start 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 0x7f10d815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 0x7f10d815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80708f0 con 0x7f10c4003a00 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80748b0 con 0x7f10d809e080 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10de0a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f10d8070a70 con 0x7f10d809e490 2024-09-21T07:27:10.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58236/0 (socket says 172.21.15.105:58236) 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 -- 172.21.15.105:0/2919049917 learned_addr learned my addr 172.21.15.105:0/2919049917 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d7fff640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 0x7f10d815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d6ffd640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 0x7f10d815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 -- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 msgr2=0x7f10d815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 0x7f10d815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 -- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 msgr2=0x7f10d815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:10.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 0x7f10d815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d77fe640 1 -- 172.21.15.105:0/2919049917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d81f6740 con 0x7f10c4003a00 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d6ffd640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 0x7f10d815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.522+0000 7f10d7fff640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 0x7f10d815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10d77fe640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10c8036040 tx=0x7f10c8033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8038040 con 0x7f10c4003a00 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10c802bc10 con 0x7f10c4003a00 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004b20 con 0x7f10c4003a00 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d81f6a20 con 0x7f10c4003a00 2024-09-21T07:27:10.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.523+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d81f6fe0 con 0x7f10c4003a00 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.524+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d806cf80 con 0x7f10c4003a00 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.525+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10c8045020 con 0x7f10c4003a00 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.525+0000 7f10cffff640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 0x7f10b807b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.525+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7f10c80bc620 con 0x7f10c4003a00 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.526+0000 7f10d6ffd640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 0x7f10b807b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:10.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.526+0000 7f10d6ffd640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 0x7f10b807b0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10d806dbb0 tx=0x7f10c0005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:10.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.530+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10c8084a10 con 0x7f10c4003a00 2024-09-21T07:27:10.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:10.741+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f10d80aeba0 con 0x7f10c4003a00 2024-09-21T07:27:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.398+0000 7f10cffff640 1 -- 172.21.15.105:0/2919049917 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f10c8084310 con 0x7f10c4003a00 2024-09-21T07:27:11.400 INFO:teuthology.orchestra.run.smithi105.stderr:new fs with metadata pool 5 and data pool 6 2024-09-21T07:27:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.405+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 msgr2=0x7f10b807b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.405+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 0x7f10b807b0c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10d806dbb0 tx=0x7f10c0005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.405+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 msgr2=0x7f10d8156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.405+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10c8036040 tx=0x7f10c8033c10 comp rx=0 tx=0).stop 2024-09-21T07:27:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 shutdown_connections 2024-09-21T07:27:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f10b8078bb0 0x7f10b807b0c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10d809e080 0x7f10d815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f10d809e490 0x7f10d815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 --2- 172.21.15.105:0/2919049917 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f10c4003a00 0x7f10d8156a40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.407+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 >> 172.21.15.105:0/2919049917 conn(0x7f10d80cdce0 msgr2=0x7f10d80b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.408+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 shutdown_connections 2024-09-21T07:27:11.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.408+0000 7f10de0a9640 1 -- 172.21.15.105:0/2919049917 wait complete. 2024-09-21T07:27:11.430 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 -- 172.21.15.105:0/64929291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80cb4a0 msgr2=0x7fbdf8073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 --2- 172.21.15.105:0/64929291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80cb4a0 0x7fbdf8073dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009830 tx=0x7fbde802a510 comp rx=0 tx=0).stop 2024-09-21T07:27:11.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 -- 172.21.15.105:0/64929291 shutdown_connections 2024-09-21T07:27:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 --2- 172.21.15.105:0/64929291 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cc1d0 0x7fbdf80cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 --2- 172.21.15.105:0/64929291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80cb4a0 0x7fbdf8073dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 -- 172.21.15.105:0/64929291 >> 172.21.15.105:0/64929291 conn(0x7fbdf80ccb90 msgr2=0x7fbdf80ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 -- 172.21.15.105:0/64929291 shutdown_connections 2024-09-21T07:27:11.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.542+0000 7fbdfce94640 1 -- 172.21.15.105:0/64929291 wait complete. 2024-09-21T07:27:11.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.544+0000 7fbdfce94640 1 Processor -- start 2024-09-21T07:27:11.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.544+0000 7fbdfce94640 1 -- start start 2024-09-21T07:27:11.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.545+0000 7fbdfce94640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80ca760 0x7fbdf809aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.545+0000 7fbdfce94640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cb4a0 0x7fbdf814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.545+0000 7fbdfce94640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdfce94640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80ceb00 con 0x7fbdf80c4160 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdfce94640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf80ce980 con 0x7fbdf80ca760 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdfce94640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fbdf80cec80 con 0x7fbdf80cb4a0 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58238/0 (socket says 172.21.15.105:58238) 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80ca760 0x7fbdf809aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 -- 172.21.15.105:0/4218932810 learned_addr learned my addr 172.21.15.105:0/4218932810 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf5d9b640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cb4a0 0x7fbdf814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 -- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cb4a0 msgr2=0x7fbdf814da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cb4a0 0x7fbdf814da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 -- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80ca760 msgr2=0x7fbdf809aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80ca760 0x7fbdf809aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 -- 172.21.15.105:0/4218932810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf814df80 con 0x7fbdf80c4160 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdf6d9d640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000efc0 tx=0x7fbdf000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0003970 con 0x7fbdf80c4160 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdf0003c00 con 0x7fbdf80c4160 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0004020 con 0x7fbdf80c4160 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf8150e30 con 0x7fbdf80c4160 2024-09-21T07:27:11.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.546+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf809c540 con 0x7fbdf80c4160 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.547+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf809afe0 con 0x7fbdf80c4160 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.548+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbdf001d020 con 0x7fbdf80c4160 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.549+0000 7fbde6ffd640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.549+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbdf009b410 con 0x7fbdf80c4160 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.549+0000 7fbdf659c640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.549+0000 7fbdf659c640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 0x7fbdd807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbde8034000 tx=0x7fbde8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.553+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbdf809afe0 con 0x7fbdf80c4160 2024-09-21T07:27:11.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.730+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdf80cf130 con 0x7fbdf80c4160 2024-09-21T07:27:11.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.731+0000 7fbde6ffd640 1 -- 172.21.15.105:0/4218932810 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17087 (secure 0 0 0) 0x7fbdf006ae20 con 0x7fbdf80c4160 2024-09-21T07:27:11.733 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:11.733 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:10.743196+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:27:08.726593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T07:27:09.704686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 msgr2=0x7fbdd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.735 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 0x7fbdd807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbde8034000 tx=0x7fbde8004730 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 msgr2=0x7fbdf80b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000efc0 tx=0x7fbdf000c800 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 shutdown_connections 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fbdd8078b90 0x7fbdd807b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbdf80c4160 0x7fbdf80b51b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbdf80cb4a0 0x7fbdf814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 --2- 172.21.15.105:0/4218932810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdf80ca760 0x7fbdf809aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 >> 172.21.15.105:0/4218932810 conn(0x7fbdf80ccb90 msgr2=0x7fbdf8068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 shutdown_connections 2024-09-21T07:27:11.736 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.733+0000 7fbdfce94640 1 -- 172.21.15.105:0/4218932810 wait complete. 2024-09-21T07:27:11.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:11.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 -- 172.21.15.105:0/303443662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e40d0bb0 msgr2=0x7f05e40d0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 --2- 172.21.15.105:0/303443662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e40d0bb0 0x7f05e40d0fc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f05dc005fd0 tx=0x7f05dc02a840 comp rx=0 tx=0).stop 2024-09-21T07:27:11.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 -- 172.21.15.105:0/303443662 shutdown_connections 2024-09-21T07:27:11.859 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 --2- 172.21.15.105:0/303443662 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e40d0bb0 0x7f05e40d0fc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 --2- 172.21.15.105:0/303443662 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 -- 172.21.15.105:0/303443662 >> 172.21.15.105:0/303443662 conn(0x7f05e406dd60 msgr2=0x7f05e406e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:11.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 -- 172.21.15.105:0/303443662 shutdown_connections 2024-09-21T07:27:11.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.857+0000 7f05e999d640 1 -- 172.21.15.105:0/303443662 wait complete. 2024-09-21T07:27:11.860 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.859+0000 7f05e999d640 1 Processor -- start 2024-09-21T07:27:11.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.859+0000 7f05e999d640 1 -- start start 2024-09-21T07:27:11.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 0x7f05e40ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e41f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05e40cf540 con 0x7f05e40d0bb0 2024-09-21T07:27:11.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05e40cf3c0 con 0x7f05e415eca0 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e999d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f05e40cf6c0 con 0x7f05e4162d90 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e899b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e899b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58248/0 (socket says 172.21.15.105:58248) 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.860+0000 7f05e899b640 1 -- 172.21.15.105:0/2547559129 learned_addr learned my addr 172.21.15.105:0/2547559129 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e919c640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e41f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e3fff640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 0x7f05e40ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 -- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 msgr2=0x7f05e41f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e41f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 -- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 msgr2=0x7f05e40ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 0x7f05e40ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 -- 172.21.15.105:0/2547559129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e41f5270 con 0x7f05e40d0bb0 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e919c640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e41f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e3fff640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 0x7f05e40ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:11.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e899b640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05d400e790 tx=0x7f05d400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d400eea0 con 0x7f05e40d0bb0 2024-09-21T07:27:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05d400cdb0 con 0x7f05e40d0bb0 2024-09-21T07:27:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e40cd650 con 0x7f05e40d0bb0 2024-09-21T07:27:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4010690 con 0x7f05e40d0bb0 2024-09-21T07:27:11.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.861+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e41f57d0 con 0x7f05e40d0bb0 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.862+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e40cfbb0 con 0x7f05e40d0bb0 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.863+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05d4010830 con 0x7f05e40d0bb0 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.863+0000 7f05e17fa640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 0x7f05d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.863+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f05d409b9a0 con 0x7f05e40d0bb0 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.864+0000 7f05e3fff640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 0x7f05d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:11.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.864+0000 7f05e3fff640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 0x7f05d007b0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05e41529c0 tx=0x7f05d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:11.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:11.868+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05e41f5270 con 0x7f05e40d0bb0 2024-09-21T07:27:12.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.044+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05e406e4f0 con 0x7f05e40d0bb0 2024-09-21T07:27:12.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.045+0000 7f05e17fa640 1 -- 172.21.15.105:0/2547559129 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17087 (secure 0 0 0) 0x7f05d4063bd0 con 0x7f05e40d0bb0 2024-09-21T07:27:12.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:12.048 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":40,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:10.743196+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:27:08.726593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T07:27:09.704686+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:12.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.047+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 msgr2=0x7f05d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.047+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 0x7f05d007b0d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f05e41529c0 tx=0x7f05d8009040 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.047+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 msgr2=0x7f05e40cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.047+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f05d400e790 tx=0x7f05d400ec50 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 shutdown_connections 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f05d0078bc0 0x7f05d007b0d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05e415eca0 0x7f05e40ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05e4162d90 0x7f05e41f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 --2- 172.21.15.105:0/2547559129 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05e40d0bb0 0x7f05e40cd110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 >> 172.21.15.105:0/2547559129 conn(0x7f05e406dd60 msgr2=0x7f05e4161e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 shutdown_connections 2024-09-21T07:27:12.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.048+0000 7f05e999d640 1 -- 172.21.15.105:0/2547559129 wait complete. 2024-09-21T07:27:12.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:27:12.176 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 -- 172.21.15.105:0/3451446756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 msgr2=0x7f01f80d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 --2- 172.21.15.105:0/3451446756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80d4870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f01e0005fd0 tx=0x7f01e002a540 comp rx=0 tx=0).stop 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 -- 172.21.15.105:0/3451446756 shutdown_connections 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 --2- 172.21.15.105:0/3451446756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80d4870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 --2- 172.21.15.105:0/3451446756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d3020 0x7f01f80b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 -- 172.21.15.105:0/3451446756 >> 172.21.15.105:0/3451446756 conn(0x7f01f808bfb0 msgr2=0x7f01f809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.174+0000 7f01fe562640 1 -- 172.21.15.105:0/3451446756 shutdown_connections 2024-09-21T07:27:12.177 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.175+0000 7f01fe562640 1 -- 172.21.15.105:0/3451446756 wait complete. 2024-09-21T07:27:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.176+0000 7f01fe562640 1 Processor -- start 2024-09-21T07:27:12.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.176+0000 7f01fe562640 1 -- start start 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d0fc0 0x7f01f81f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80d01c0 con 0x7f01f80d3020 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01f80b4b30 con 0x7f01f80d4460 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fe562640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f01f80d0340 con 0x7f01f80d0fc0 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58262/0 (socket says 172.21.15.105:58262) 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 -- 172.21.15.105:0/35856704 learned_addr learned my addr 172.21.15.105:0/35856704 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01fcaff640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:12.181 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f7fff640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d0fc0 0x7f01f81f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 -- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d0fc0 msgr2=0x7f01f81f6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d0fc0 0x7f01f81f6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 -- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 msgr2=0x7f01f80c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 -- 172.21.15.105:0/35856704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f80c9320 con 0x7f01f80d3020 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.178+0000 7f01f77fe640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01e0034040 tx=0x7f01e00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.179+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e003e030 con 0x7f01f80d3020 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.179+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01e0003d90 con 0x7f01f80d3020 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.179+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f815ca90 con 0x7f01f80d3020 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.179+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e0035040 con 0x7f01f80d3020 2024-09-21T07:27:12.182 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.179+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f80b3ed0 con 0x7f01f80d3020 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.180+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f80d1bc0 con 0x7f01f80d3020 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.180+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01e0041030 con 0x7f01f80d3020 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.181+0000 7f01f4ff9640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 0x7f01d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.181+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f01e00bb6e0 con 0x7f01f80d3020 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.181+0000 7f01f7fff640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 0x7f01d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:12.183 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.182+0000 7f01f7fff640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 0x7f01d807b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009c00 tx=0x7f01e8005f30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:12.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.186+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01e00832c0 con 0x7f01f80d3020 2024-09-21T07:27:12.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:12.401+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f01f8162030 con 0x7f01f80d3020 2024-09-21T07:27:12.522 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:27:12.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:27:13.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.407+0000 7f01f4ff9640 1 -- 172.21.15.105:0/35856704 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f01e0087cf0 con 0x7f01f80d3020 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 msgr2=0x7f01d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 0x7f01d807b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009c00 tx=0x7f01e8005f30 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 msgr2=0x7f01f815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f01e0034040 tx=0x7f01e00038a0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 shutdown_connections 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f01d8078b90 0x7f01d807b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01f80d4460 0x7f01f80c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f01f80d3020 0x7f01f815c550 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 --2- 172.21.15.105:0/35856704 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01f80d0fc0 0x7f01f81f6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 >> 172.21.15.105:0/35856704 conn(0x7f01f808bfb0 msgr2=0x7f01f806de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 shutdown_connections 2024-09-21T07:27:13.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.410+0000 7f01fe562640 1 -- 172.21.15.105:0/35856704 wait complete. 2024-09-21T07:27:13.426 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:27:13.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:27:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 -- 172.21.15.105:0/1853889729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40c5a50 msgr2=0x7f98c40d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/1853889729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40c5a50 0x7f98c40d0170 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f98c40c92a0 tx=0x7f98bc02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 -- 172.21.15.105:0/1853889729 shutdown_connections 2024-09-21T07:27:13.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/1853889729 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40b1fd0 0x7f98c40ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/1853889729 >> v2:172.21.15.112:3300/0 conn(0x7f98c40d06b0 0x7f98c40b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/1853889729 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40c5a50 0x7f98c40d0170 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 -- 172.21.15.105:0/1853889729 >> 172.21.15.105:0/1853889729 conn(0x7f98c40cee40 msgr2=0x7f98c40cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 -- 172.21.15.105:0/1853889729 shutdown_connections 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.546+0000 7f98cb2b6640 1 -- 172.21.15.105:0/1853889729 wait complete. 2024-09-21T07:27:13.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.548+0000 7f98cb2b6640 1 Processor -- start 2024-09-21T07:27:13.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.548+0000 7f98cb2b6640 1 -- start start 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40c5a50 0x7f98c415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c40d06b0 0x7f98c40c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40cebd0 con 0x7f98c40b1fd0 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40cea50 con 0x7f98c40d06b0 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98cb2b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f98c409bc80 con 0x7f98c40c5a50 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98c9052640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:13.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98c9052640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58276/0 (socket says 172.21.15.105:58276) 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98c9052640 1 -- 172.21.15.105:0/3138495610 learned_addr learned my addr 172.21.15.105:0/3138495610 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98c8851640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40c5a50 0x7f98c415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.549+0000 7f98c9853640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c40d06b0 0x7f98c40c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 -- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40c5a50 msgr2=0x7f98c415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40c5a50 0x7f98c415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 -- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c40d06b0 msgr2=0x7f98c40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c40d06b0 0x7f98c40c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 -- 172.21.15.105:0/3138495610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41f5550 con 0x7f98c40b1fd0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c9052640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f98bc02aaa0 tx=0x7f98bc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc033e60 con 0x7f98c40b1fd0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98bc031950 con 0x7f98c40b1fd0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c41f5830 con 0x7f98c40b1fd0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98bc031d80 con 0x7f98c40b1fd0 2024-09-21T07:27:13.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.550+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c41f5df0 con 0x7f98c40b1fd0 2024-09-21T07:27:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.551+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c40d0170 con 0x7f98c40b1fd0 2024-09-21T07:27:13.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.552+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98bc005040 con 0x7f98c40b1fd0 2024-09-21T07:27:13.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.553+0000 7f98c1ffb640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:13.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.553+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f98bc0bbe70 con 0x7f98c40b1fd0 2024-09-21T07:27:13.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.553+0000 7f98c8851640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:13.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.554+0000 7f98c8851640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 0x7f98b007b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c41f8900 tx=0x7f98b8005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:13.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.558+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98bc03f030 con 0x7f98c40b1fd0 2024-09-21T07:27:13.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:13.772+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f98c40cf3d0 con 0x7f98c40b1fd0 2024-09-21T07:27:14.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.408+0000 7f98c1ffb640 1 -- 172.21.15.105:0/3138495610 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f98bc031770 con 0x7f98c40b1fd0 2024-09-21T07:27:14.412 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.410+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 msgr2=0x7f98b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.410+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 0x7f98b007b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98c41f8900 tx=0x7f98b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.410+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 msgr2=0x7f98c40cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.410+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f98bc02aaa0 tx=0x7f98bc0049d0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 shutdown_connections 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98b0078b90 0x7f98b007b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c40d06b0 0x7f98c40c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c40c5a50 0x7f98c415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 --2- 172.21.15.105:0/3138495610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98c40b1fd0 0x7f98c40cbb40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 >> 172.21.15.105:0/3138495610 conn(0x7f98c40cee40 msgr2=0x7f98c40b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 shutdown_connections 2024-09-21T07:27:14.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.411+0000 7f98cb2b6640 1 -- 172.21.15.105:0/3138495610 wait complete. 2024-09-21T07:27:14.425 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:27:14.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2152566793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 msgr2=0x7f6ba0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2152566793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6ba0004800 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80b3eb0 tx=0x7f6ba0005830 comp rx=0 tx=0).stop 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbdc7b640 1 -- 172.21.15.105:0/2152566793 reap_dead start 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2152566793 shutdown_connections 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2152566793 >> 172.21.15.105:0/2152566793 conn(0x7f6bb809d850 msgr2=0x7f6bb808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2152566793 shutdown_connections 2024-09-21T07:27:14.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.541+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2152566793 wait complete. 2024-09-21T07:27:14.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.542+0000 7f6bbf6de640 1 Processor -- start 2024-09-21T07:27:14.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.543+0000 7f6bbf6de640 1 -- start start 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bb80cdbb0 0x7f6bb80cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 0x7f6bb80b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80cf0e0 con 0x7f6ba0004080 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb80cef60 con 0x7f6bb80cdbb0 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb80cf260 con 0x7f6bb80b25b0 2024-09-21T07:27:14.547 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58292/0 (socket says 172.21.15.105:58292) 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 -- 172.21.15.105:0/2562036895 learned_addr learned my addr 172.21.15.105:0/2562036895 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbdc7b640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 0x7f6bb80b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbcc79640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bb80cdbb0 0x7f6bb80cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 -- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 msgr2=0x7f6bb80b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 0x7f6bb80b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 -- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bb80cdbb0 msgr2=0x7f6bb80cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bb80cdbb0 0x7f6bb80cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 -- 172.21.15.105:0/2562036895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb80b2f00 con 0x7f6ba0004080 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbdc7b640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 0x7f6bb80b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbd47a640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001550 tx=0x7f6ba80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8015070 con 0x7f6ba0004080 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ba8011900 con 0x7f6ba0004080 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb81f8bc0 con 0x7f6ba0004080 2024-09-21T07:27:14.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.544+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8011d50 con 0x7f6ba0004080 2024-09-21T07:27:14.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.545+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81f9140 con 0x7f6ba0004080 2024-09-21T07:27:14.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.546+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bb806f040 con 0x7f6ba0004080 2024-09-21T07:27:14.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.546+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ba8018070 con 0x7f6ba0004080 2024-09-21T07:27:14.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.547+0000 7f6bb5ffb640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 0x7f6ba407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.547+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f6ba809bd10 con 0x7f6ba0004080 2024-09-21T07:27:14.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.547+0000 7f6bbcc79640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 0x7f6ba407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.548+0000 7f6bbcc79640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 0x7f6ba407b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bac036000 tx=0x7f6bac0046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:14.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.552+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6bb80b2f00 con 0x7f6ba0004080 2024-09-21T07:27:14.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.763+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bb80b8450 con 0x7f6ba0004080 2024-09-21T07:27:14.766 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.764+0000 7f6bb5ffb640 1 -- 172.21.15.105:0/2562036895 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7f6ba8063fd0 con 0x7f6ba0004080 2024-09-21T07:27:14.766 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:14.766 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:14.767 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 msgr2=0x7f6ba407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 0x7f6ba407b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6bac036000 tx=0x7f6bac0046c0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 msgr2=0x7f6bb815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8001550 tx=0x7f6ba80039c0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 shutdown_connections 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f6ba4078bc0 0x7f6ba407b0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.769 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bb80cdbb0 0x7f6bb80cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6bb80b25b0 0x7f6bb80b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 --2- 172.21.15.105:0/2562036895 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6ba0004080 0x7f6bb815ea20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 >> 172.21.15.105:0/2562036895 conn(0x7f6bb809d850 msgr2=0x7f6bb81619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 shutdown_connections 2024-09-21T07:27:14.770 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.766+0000 7f6bbf6de640 1 -- 172.21.15.105:0/2562036895 wait complete. 2024-09-21T07:27:14.780 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 -- 172.21.15.105:0/4216256349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe890003a50 msgr2=0x7fe890003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 --2- 172.21.15.105:0/4216256349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe890003a50 0x7fe890003ec0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe8980013d0 tx=0x7fe890005830 comp rx=0 tx=0).stop 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 -- 172.21.15.105:0/4216256349 shutdown_connections 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 --2- 172.21.15.105:0/4216256349 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe890003a50 0x7fe890003ec0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 -- 172.21.15.105:0/4216256349 >> 172.21.15.105:0/4216256349 conn(0x7fe8a809aa20 msgr2=0x7fe8a80b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 -- 172.21.15.105:0/4216256349 shutdown_connections 2024-09-21T07:27:14.893 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.891+0000 7fe8acb01640 1 -- 172.21.15.105:0/4216256349 wait complete. 2024-09-21T07:27:14.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.893+0000 7fe8acb01640 1 Processor -- start 2024-09-21T07:27:14.894 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.893+0000 7fe8acb01640 1 -- start start 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe890003a50 0x7fe8a80b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8a8159ae0 0x7fe8a8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a815baa0 con 0x7fe8a8149010 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a815b920 con 0x7fe890003a50 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8acb01640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a80b0290 con 0x7fe8a8159ae0 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58308/0 (socket says 172.21.15.105:58308) 2024-09-21T07:27:14.897 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 -- 172.21.15.105:0/2095376234 learned_addr learned my addr 172.21.15.105:0/2095376234 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a659c640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe890003a50 0x7fe8a80b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a6d9d640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8a8159ae0 0x7fe8a8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 -- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8a8159ae0 msgr2=0x7fe8a8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8a8159ae0 0x7fe8a8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 -- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe890003a50 msgr2=0x7fe8a80b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe890003a50 0x7fe8a80b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 -- 172.21.15.105:0/2095376234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a8159ef0 con 0x7fe8a8149010 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.894+0000 7fe8a5d9b640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe89c006690 tx=0x7fe89c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.895+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c015020 con 0x7fe8a8149010 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.895+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe89c011900 con 0x7fe8a8149010 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.895+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81f9380 con 0x7fe8a8149010 2024-09-21T07:27:14.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.895+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89c011d50 con 0x7fe8a8149010 2024-09-21T07:27:14.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.895+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81f9940 con 0x7fe8a8149010 2024-09-21T07:27:14.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.896+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a80b4af0 con 0x7fe8a8149010 2024-09-21T07:27:14.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.897+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe89c018020 con 0x7fe8a8149010 2024-09-21T07:27:14.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.897+0000 7fe896ffd640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:14.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.897+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe89c09b810 con 0x7fe8a8149010 2024-09-21T07:27:14.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.897+0000 7fe8a659c640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:14.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.898+0000 7fe8a659c640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 0x7fe88807b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe898033000 tx=0x7fe89802d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:14.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:14.902+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe89c01f020 con 0x7fe8a8149010 2024-09-21T07:27:15.082 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.080+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8a80b07a0 con 0x7fe8a8149010 2024-09-21T07:27:15.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.081+0000 7fe896ffd640 1 -- 172.21.15.105:0/2095376234 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17074 (secure 0 0 0) 0x7fe8a8159ef0 con 0x7fe8a8149010 2024-09-21T07:27:15.084 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:15.084 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":41,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:12.385932+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:27:08.726593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T07:27:09.704686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 msgr2=0x7fe88807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 0x7fe88807b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe898033000 tx=0x7fe89802d040 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 msgr2=0x7fe8a81595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe89c006690 tx=0x7fe89c0016f0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 shutdown_connections 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fe888078b90 0x7fe88807b0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe8a8159ae0 0x7fe8a8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe8a8149010 0x7fe8a81595a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 --2- 172.21.15.105:0/2095376234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe890003a50 0x7fe8a80b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 >> 172.21.15.105:0/2095376234 conn(0x7fe8a809aa20 msgr2=0x7fe8a80b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 shutdown_connections 2024-09-21T07:27:15.087 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.084+0000 7fe8acb01640 1 -- 172.21.15.105:0/2095376234 wait complete. 2024-09-21T07:27:15.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:27:15.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2952092012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 msgr2=0x7fc84000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2952092012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc84000b9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc850001690 tx=0x7fc840005830 comp rx=0 tx=0).stop 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2952092012 shutdown_connections 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2952092012 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc84000b9c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2952092012 >> 172.21.15.105:0/2952092012 conn(0x7fc8580cedb0 msgr2=0x7fc8580cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2952092012 shutdown_connections 2024-09-21T07:27:15.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.209+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2952092012 wait complete. 2024-09-21T07:27:15.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.210+0000 7fc85f4d5640 1 Processor -- start 2024-09-21T07:27:15.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.211+0000 7fc85f4d5640 1 -- start start 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8580c93e0 0x7fc8581534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8581597f0 0x7fc858157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580d0490 con 0x7fc840005250 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc8580d0310 con 0x7fc8580c93e0 2024-09-21T07:27:15.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85f4d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc8580d0610 con 0x7fc8581597f0 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58314/0 (socket says 172.21.15.105:58314) 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 -- 172.21.15.105:0/2060151283 learned_addr learned my addr 172.21.15.105:0/2060151283 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85da72640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8581597f0 0x7fc858157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85ca70640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8580c93e0 0x7fc8581534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 -- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8581597f0 msgr2=0x7fc858157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8581597f0 0x7fc858157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 -- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8580c93e0 msgr2=0x7fc8581534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8580c93e0 0x7fc8581534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 -- 172.21.15.105:0/2060151283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8581f43c0 con 0x7fc840005250 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.212+0000 7fc85d271640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc848001c20 tx=0x7fc8480066e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.213+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848015070 con 0x7fc840005250 2024-09-21T07:27:15.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.213+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc84800fad0 con 0x7fc840005250 2024-09-21T07:27:15.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.213+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8580cbc90 con 0x7fc840005250 2024-09-21T07:27:15.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.213+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848011430 con 0x7fc840005250 2024-09-21T07:27:15.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.213+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc858158d10 con 0x7fc840005250 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.214+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8580d0fb0 con 0x7fc840005250 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.215+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8480115d0 con 0x7fc840005250 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.215+0000 7fc855ffb640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 0x7fc84407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.215+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fc84809bcb0 con 0x7fc840005250 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.215+0000 7fc85ca70640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 0x7fc84407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.216+0000 7fc85ca70640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 0x7fc84407b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc8580cb280 tx=0x7fc84c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.220+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8581f43c0 con 0x7fc840005250 2024-09-21T07:27:15.354 INFO:tasks.ceph.mds.a.smithi105.stderr:2024-09-21T07:27:15.351+0000 7fc5e9af6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:27:15.394 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.392+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8580b3c00 con 0x7fc844078bc0 2024-09-21T07:27:15.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.397+0000 7fc855ffb640 1 -- 172.21.15.105:0/2060151283 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fc8580b3c00 con 0x7fc844078bc0 2024-09-21T07:27:15.401 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:15.403 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 msgr2=0x7fc84407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 0x7fc84407b0d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc8580cb280 tx=0x7fc84c005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 msgr2=0x7fc8581f3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc848001c20 tx=0x7fc8480066e0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 shutdown_connections 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fc844078bc0 0x7fc84407b0d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc8580c93e0 0x7fc8581534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc8581597f0 0x7fc858157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 --2- 172.21.15.105:0/2060151283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc840005250 0x7fc8581f3e80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 >> 172.21.15.105:0/2060151283 conn(0x7fc8580cedb0 msgr2=0x7fc8580b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 shutdown_connections 2024-09-21T07:27:15.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.400+0000 7fc85f4d5640 1 -- 172.21.15.105:0/2060151283 wait complete. 2024-09-21T07:27:15.414 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-21T07:27:14.780401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221084,"kb_used_data":4844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753636,"statfs":{"total":773094113280,"available":772867723264,"internally_reserved":0,"allocated":4960256,"data_stored":2806554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407869"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.405331+0000","last_change":"2024-09-21T07:27:11.400818+0000","last_active":"2024-09-21T07:27:11.405331+0000","last_peered":"2024-09-21T07:27:11.405331+0000","last_clean":"2024-09-21T07:27:11.405331+0000","last_became_active":"2024-09-21T07:27:11.400518+0000","last_became_peered":"2024-09-21T07:27:11.400518+0000","last_unstale":"2024-09-21T07:27:11.405331+0000","last_undegraded":"2024-09-21T07:27:11.405331+0000","last_fullsized":"2024-09-21T07:27:11.405331+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:35.470013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.404264+0000","last_change":"2024-09-21T07:27:11.399663+0000","last_active":"2024-09-21T07:27:11.404264+0000","last_peered":"2024-09-21T07:27:11.404264+0000","last_clean":"2024-09-21T07:27:11.404264+0000","last_became_active":"2024-09-21T07:27:11.399345+0000","last_became_peered":"2024-09-21T07:27:11.399345+0000","last_unstale":"2024-09-21T07:27:11.404264+0000","last_undegraded":"2024-09-21T07:27:11.404264+0000","last_fullsized":"2024-09-21T07:27:11.404264+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:16.101562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156904+0000","last_change":"2024-09-21T07:27:11.420097+0000","last_active":"2024-09-21T07:27:13.156904+0000","last_peered":"2024-09-21T07:27:13.156904+0000","last_clean":"2024-09-21T07:27:13.156904+0000","last_became_active":"2024-09-21T07:27:11.419711+0000","last_became_peered":"2024-09-21T07:27:11.419711+0000","last_unstale":"2024-09-21T07:27:13.156904+0000","last_undegraded":"2024-09-21T07:27:13.156904+0000","last_fullsized":"2024-09-21T07:27:13.156904+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:57.840508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406514+0000","last_change":"2024-09-21T07:27:11.412520+0000","last_active":"2024-09-21T07:27:12.406514+0000","last_peered":"2024-09-21T07:27:12.406514+0000","last_clean":"2024-09-21T07:27:12.406514+0000","last_became_active":"2024-09-21T07:27:11.412196+0000","last_became_peered":"2024-09-21T07:27:11.412196+0000","last_unstale":"2024-09-21T07:27:12.406514+0000","last_undegraded":"2024-09-21T07:27:12.406514+0000","last_fullsized":"2024-09-21T07:27:12.406514+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:04.293157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406422+0000","last_change":"2024-09-21T07:27:11.412623+0000","last_active":"2024-09-21T07:27:12.406422+0000","last_peered":"2024-09-21T07:27:12.406422+0000","last_clean":"2024-09-21T07:27:12.406422+0000","last_became_active":"2024-09-21T07:27:11.412142+0000","last_became_peered":"2024-09-21T07:27:11.412142+0000","last_unstale":"2024-09-21T07:27:12.406422+0000","last_undegraded":"2024-09-21T07:27:12.406422+0000","last_fullsized":"2024-09-21T07:27:12.406422+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.215621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157363+0000","last_change":"2024-09-21T07:27:11.406389+0000","last_active":"2024-09-21T07:27:13.157363+0000","last_peered":"2024-09-21T07:27:13.157363+0000","last_clean":"2024-09-21T07:27:13.157363+0000","last_became_active":"2024-09-21T07:27:11.406082+0000","last_became_peered":"2024-09-21T07:27:11.406082+0000","last_unstale":"2024-09-21T07:27:13.157363+0000","last_undegraded":"2024-09-21T07:27:13.157363+0000","last_fullsized":"2024-09-21T07:27:13.157363+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:51.487806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408798+0000","last_change":"2024-09-21T07:27:11.423122+0000","last_active":"2024-09-21T07:27:12.408798+0000","last_peered":"2024-09-21T07:27:12.408798+0000","last_clean":"2024-09-21T07:27:12.408798+0000","last_became_active":"2024-09-21T07:27:11.421902+0000","last_became_peered":"2024-09-21T07:27:11.421902+0000","last_unstale":"2024-09-21T07:27:12.408798+0000","last_undegraded":"2024-09-21T07:27:12.408798+0000","last_fullsized":"2024-09-21T07:27:12.408798+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:13.731711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420887+0000","last_change":"2024-09-21T07:27:11.418117+0000","last_active":"2024-09-21T07:27:11.420887+0000","last_peered":"2024-09-21T07:27:11.420887+0000","last_clean":"2024-09-21T07:27:11.420887+0000","last_became_active":"2024-09-21T07:27:11.417686+0000","last_became_peered":"2024-09-21T07:27:11.417686+0000","last_unstale":"2024-09-21T07:27:11.420887+0000","last_undegraded":"2024-09-21T07:27:11.420887+0000","last_fullsized":"2024-09-21T07:27:11.420887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:13.784958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156809+0000","last_change":"2024-09-21T07:27:11.420211+0000","last_active":"2024-09-21T07:27:13.156809+0000","last_peered":"2024-09-21T07:27:13.156809+0000","last_clean":"2024-09-21T07:27:13.156809+0000","last_became_active":"2024-09-21T07:27:11.419856+0000","last_became_peered":"2024-09-21T07:27:11.419856+0000","last_unstale":"2024-09-21T07:27:13.156809+0000","last_undegraded":"2024-09-21T07:27:13.156809+0000","last_fullsized":"2024-09-21T07:27:13.156809+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:32.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409176+0000","last_change":"2024-09-21T07:27:11.426553+0000","last_active":"2024-09-21T07:27:12.409176+0000","last_peered":"2024-09-21T07:27:12.409176+0000","last_clean":"2024-09-21T07:27:12.409176+0000","last_became_active":"2024-09-21T07:27:11.426234+0000","last_became_peered":"2024-09-21T07:27:11.426234+0000","last_unstale":"2024-09-21T07:27:12.409176+0000","last_undegraded":"2024-09-21T07:27:12.409176+0000","last_fullsized":"2024-09-21T07:27:12.409176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:30.080550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409292+0000","last_change":"2024-09-21T07:27:11.419680+0000","last_active":"2024-09-21T07:27:12.409292+0000","last_peered":"2024-09-21T07:27:12.409292+0000","last_clean":"2024-09-21T07:27:12.409292+0000","last_became_active":"2024-09-21T07:27:11.419329+0000","last_became_peered":"2024-09-21T07:27:11.419329+0000","last_unstale":"2024-09-21T07:27:12.409292+0000","last_undegraded":"2024-09-21T07:27:12.409292+0000","last_fullsized":"2024-09-21T07:27:12.409292+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:46.580975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156601+0000","last_change":"2024-09-21T07:27:11.398837+0000","last_active":"2024-09-21T07:27:13.156601+0000","last_peered":"2024-09-21T07:27:13.156601+0000","last_clean":"2024-09-21T07:27:13.156601+0000","last_became_active":"2024-09-21T07:27:11.398312+0000","last_became_peered":"2024-09-21T07:27:11.398312+0000","last_unstale":"2024-09-21T07:27:13.156601+0000","last_undegraded":"2024-09-21T07:27:13.156601+0000","last_fullsized":"2024-09-21T07:27:13.156601+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:11.081801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408854+0000","last_change":"2024-09-21T07:27:11.423489+0000","last_active":"2024-09-21T07:27:12.408854+0000","last_peered":"2024-09-21T07:27:12.408854+0000","last_clean":"2024-09-21T07:27:12.408854+0000","last_became_active":"2024-09-21T07:27:11.422541+0000","last_became_peered":"2024-09-21T07:27:11.422541+0000","last_unstale":"2024-09-21T07:27:12.408854+0000","last_undegraded":"2024-09-21T07:27:12.408854+0000","last_fullsized":"2024-09-21T07:27:12.408854+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:57.859007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156323+0000","last_change":"2024-09-21T07:27:11.409499+0000","last_active":"2024-09-21T07:27:13.156323+0000","last_peered":"2024-09-21T07:27:13.156323+0000","last_clean":"2024-09-21T07:27:13.156323+0000","last_became_active":"2024-09-21T07:27:11.407759+0000","last_became_peered":"2024-09-21T07:27:11.407759+0000","last_unstale":"2024-09-21T07:27:13.156323+0000","last_undegraded":"2024-09-21T07:27:13.156323+0000","last_fullsized":"2024-09-21T07:27:13.156323+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:53.609448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408821+0000","last_change":"2024-09-21T07:27:11.421422+0000","last_active":"2024-09-21T07:27:12.408821+0000","last_peered":"2024-09-21T07:27:12.408821+0000","last_clean":"2024-09-21T07:27:12.408821+0000","last_became_active":"2024-09-21T07:27:11.421098+0000","last_became_peered":"2024-09-21T07:27:11.421098+0000","last_unstale":"2024-09-21T07:27:12.408821+0000","last_undegraded":"2024-09-21T07:27:12.408821+0000","last_fullsized":"2024-09-21T07:27:12.408821+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:14.367091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.392890+0000","last_change":"2024-09-21T07:27:11.418588+0000","last_active":"2024-09-21T07:27:12.392890+0000","last_peered":"2024-09-21T07:27:12.392890+0000","last_clean":"2024-09-21T07:27:12.392890+0000","last_became_active":"2024-09-21T07:27:11.418249+0000","last_became_peered":"2024-09-21T07:27:11.418249+0000","last_unstale":"2024-09-21T07:27:12.392890+0000","last_undegraded":"2024-09-21T07:27:12.392890+0000","last_fullsized":"2024-09-21T07:27:12.392890+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:43.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394410+0000","last_change":"2024-09-21T07:27:11.418692+0000","last_active":"2024-09-21T07:27:12.394410+0000","last_peered":"2024-09-21T07:27:12.394410+0000","last_clean":"2024-09-21T07:27:12.394410+0000","last_became_active":"2024-09-21T07:27:11.418373+0000","last_became_peered":"2024-09-21T07:27:11.418373+0000","last_unstale":"2024-09-21T07:27:12.394410+0000","last_undegraded":"2024-09-21T07:27:12.394410+0000","last_fullsized":"2024-09-21T07:27:12.394410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:47.065755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393593+0000","last_change":"2024-09-21T07:27:11.412224+0000","last_active":"2024-09-21T07:27:12.393593+0000","last_peered":"2024-09-21T07:27:12.393593+0000","last_clean":"2024-09-21T07:27:12.393593+0000","last_became_active":"2024-09-21T07:27:11.411905+0000","last_became_peered":"2024-09-21T07:27:11.411905+0000","last_unstale":"2024-09-21T07:27:12.393593+0000","last_undegraded":"2024-09-21T07:27:12.393593+0000","last_fullsized":"2024-09-21T07:27:12.393593+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:59.686478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406679+0000","last_change":"2024-09-21T07:27:11.399147+0000","last_active":"2024-09-21T07:27:12.406679+0000","last_peered":"2024-09-21T07:27:12.406679+0000","last_clean":"2024-09-21T07:27:12.406679+0000","last_became_active":"2024-09-21T07:27:11.398841+0000","last_became_peered":"2024-09-21T07:27:11.398841+0000","last_unstale":"2024-09-21T07:27:12.406679+0000","last_undegraded":"2024-09-21T07:27:12.406679+0000","last_fullsized":"2024-09-21T07:27:12.406679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:33.399273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393137+0000","last_change":"2024-09-21T07:27:11.411784+0000","last_active":"2024-09-21T07:27:12.393137+0000","last_peered":"2024-09-21T07:27:12.393137+0000","last_clean":"2024-09-21T07:27:12.393137+0000","last_became_active":"2024-09-21T07:27:11.411445+0000","last_became_peered":"2024-09-21T07:27:11.411445+0000","last_unstale":"2024-09-21T07:27:12.393137+0000","last_undegraded":"2024-09-21T07:27:12.393137+0000","last_fullsized":"2024-09-21T07:27:12.393137+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:02.250674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409031+0000","last_change":"2024-09-21T07:27:11.415333+0000","last_active":"2024-09-21T07:27:12.409031+0000","last_peered":"2024-09-21T07:27:12.409031+0000","last_clean":"2024-09-21T07:27:12.409031+0000","last_became_active":"2024-09-21T07:27:11.415040+0000","last_became_peered":"2024-09-21T07:27:11.415040+0000","last_unstale":"2024-09-21T07:27:12.409031+0000","last_undegraded":"2024-09-21T07:27:12.409031+0000","last_fullsized":"2024-09-21T07:27:12.409031+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.169804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156738+0000","last_change":"2024-09-21T07:27:11.400287+0000","last_active":"2024-09-21T07:27:13.156738+0000","last_peered":"2024-09-21T07:27:13.156738+0000","last_clean":"2024-09-21T07:27:13.156738+0000","last_became_active":"2024-09-21T07:27:11.399860+0000","last_became_peered":"2024-09-21T07:27:11.399860+0000","last_unstale":"2024-09-21T07:27:13.156738+0000","last_undegraded":"2024-09-21T07:27:13.156738+0000","last_fullsized":"2024-09-21T07:27:13.156738+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:08.394338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393161+0000","last_change":"2024-09-21T07:27:11.409776+0000","last_active":"2024-09-21T07:27:12.393161+0000","last_peered":"2024-09-21T07:27:12.393161+0000","last_clean":"2024-09-21T07:27:12.393161+0000","last_became_active":"2024-09-21T07:27:11.408115+0000","last_became_peered":"2024-09-21T07:27:11.408115+0000","last_unstale":"2024-09-21T07:27:12.393161+0000","last_undegraded":"2024-09-21T07:27:12.393161+0000","last_fullsized":"2024-09-21T07:27:12.393161+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:08.394338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406364+0000","last_change":"2024-09-21T07:27:11.409054+0000","last_active":"2024-09-21T07:27:12.406364+0000","last_peered":"2024-09-21T07:27:12.406364+0000","last_clean":"2024-09-21T07:27:12.406364+0000","last_became_active":"2024-09-21T07:27:11.408746+0000","last_became_peered":"2024-09-21T07:27:11.408746+0000","last_unstale":"2024-09-21T07:27:12.406364+0000","last_undegraded":"2024-09-21T07:27:12.406364+0000","last_fullsized":"2024-09-21T07:27:12.406364+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:02.666412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394078+0000","last_change":"2024-09-21T07:27:11.402272+0000","last_active":"2024-09-21T07:27:12.394078+0000","last_peered":"2024-09-21T07:27:12.394078+0000","last_clean":"2024-09-21T07:27:12.394078+0000","last_became_active":"2024-09-21T07:27:11.401931+0000","last_became_peered":"2024-09-21T07:27:11.401931+0000","last_unstale":"2024-09-21T07:27:12.394078+0000","last_undegraded":"2024-09-21T07:27:12.394078+0000","last_fullsized":"2024-09-21T07:27:12.394078+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:43.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157377+0000","last_change":"2024-09-21T07:27:11.417199+0000","last_active":"2024-09-21T07:27:13.157377+0000","last_peered":"2024-09-21T07:27:13.157377+0000","last_clean":"2024-09-21T07:27:13.157377+0000","last_became_active":"2024-09-21T07:27:11.416890+0000","last_became_peered":"2024-09-21T07:27:11.416890+0000","last_unstale":"2024-09-21T07:27:13.157377+0000","last_undegraded":"2024-09-21T07:27:13.157377+0000","last_fullsized":"2024-09-21T07:27:13.157377+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:57.840508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394025+0000","last_change":"2024-09-21T07:27:11.404845+0000","last_active":"2024-09-21T07:27:12.394025+0000","last_peered":"2024-09-21T07:27:12.394025+0000","last_clean":"2024-09-21T07:27:12.394025+0000","last_became_active":"2024-09-21T07:27:11.404526+0000","last_became_peered":"2024-09-21T07:27:11.404526+0000","last_unstale":"2024-09-21T07:27:12.394025+0000","last_undegraded":"2024-09-21T07:27:12.394025+0000","last_fullsized":"2024-09-21T07:27:12.394025+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:51.487806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409211+0000","last_change":"2024-09-21T07:27:11.423871+0000","last_active":"2024-09-21T07:27:12.409211+0000","last_peered":"2024-09-21T07:27:12.409211+0000","last_clean":"2024-09-21T07:27:12.409211+0000","last_became_active":"2024-09-21T07:27:11.421863+0000","last_became_peered":"2024-09-21T07:27:11.421863+0000","last_unstale":"2024-09-21T07:27:12.409211+0000","last_undegraded":"2024-09-21T07:27:12.409211+0000","last_fullsized":"2024-09-21T07:27:12.409211+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:05.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393237+0000","last_change":"2024-09-21T07:27:11.401854+0000","last_active":"2024-09-21T07:27:12.393237+0000","last_peered":"2024-09-21T07:27:12.393237+0000","last_clean":"2024-09-21T07:27:12.393237+0000","last_became_active":"2024-09-21T07:27:11.401522+0000","last_became_peered":"2024-09-21T07:27:11.401522+0000","last_unstale":"2024-09-21T07:27:12.393237+0000","last_undegraded":"2024-09-21T07:27:12.393237+0000","last_fullsized":"2024-09-21T07:27:12.393237+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:10.633067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393871+0000","last_change":"2024-09-21T07:27:11.407795+0000","last_active":"2024-09-21T07:27:12.393871+0000","last_peered":"2024-09-21T07:27:12.393871+0000","last_clean":"2024-09-21T07:27:12.393871+0000","last_became_active":"2024-09-21T07:27:11.407441+0000","last_became_peered":"2024-09-21T07:27:11.407441+0000","last_unstale":"2024-09-21T07:27:12.393871+0000","last_undegraded":"2024-09-21T07:27:12.393871+0000","last_fullsized":"2024-09-21T07:27:12.393871+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393418+0000","last_change":"2024-09-21T07:27:11.410069+0000","last_active":"2024-09-21T07:27:12.393418+0000","last_peered":"2024-09-21T07:27:12.393418+0000","last_clean":"2024-09-21T07:27:12.393418+0000","last_became_active":"2024-09-21T07:27:11.408126+0000","last_became_peered":"2024-09-21T07:27:11.408126+0000","last_unstale":"2024-09-21T07:27:12.393418+0000","last_undegraded":"2024-09-21T07:27:12.393418+0000","last_fullsized":"2024-09-21T07:27:12.393418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:14.321600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157499+0000","last_change":"2024-09-21T07:27:11.400123+0000","last_active":"2024-09-21T07:27:13.157499+0000","last_peered":"2024-09-21T07:27:13.157499+0000","last_clean":"2024-09-21T07:27:13.157499+0000","last_became_active":"2024-09-21T07:27:11.399796+0000","last_became_peered":"2024-09-21T07:27:11.399796+0000","last_unstale":"2024-09-21T07:27:13.157499+0000","last_undegraded":"2024-09-21T07:27:13.157499+0000","last_fullsized":"2024-09-21T07:27:13.157499+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394037+0000","last_change":"2024-09-21T07:27:11.418607+0000","last_active":"2024-09-21T07:27:12.394037+0000","last_peered":"2024-09-21T07:27:12.394037+0000","last_clean":"2024-09-21T07:27:12.394037+0000","last_became_active":"2024-09-21T07:27:11.418311+0000","last_became_peered":"2024-09-21T07:27:11.418311+0000","last_unstale":"2024-09-21T07:27:12.394037+0000","last_undegraded":"2024-09-21T07:27:12.394037+0000","last_fullsized":"2024-09-21T07:27:12.394037+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:14.466327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394061+0000","last_change":"2024-09-21T07:27:10.389895+0000","last_active":"2024-09-21T07:27:12.394061+0000","last_peered":"2024-09-21T07:27:12.394061+0000","last_clean":"2024-09-21T07:27:12.394061+0000","last_became_active":"2024-09-21T07:27:10.389570+0000","last_became_peered":"2024-09-21T07:27:10.389570+0000","last_unstale":"2024-09-21T07:27:12.394061+0000","last_undegraded":"2024-09-21T07:27:12.394061+0000","last_fullsized":"2024-09-21T07:27:12.394061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.319182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157555+0000","last_change":"2024-09-21T07:27:11.419977+0000","last_active":"2024-09-21T07:27:13.157555+0000","last_peered":"2024-09-21T07:27:13.157555+0000","last_clean":"2024-09-21T07:27:13.157555+0000","last_became_active":"2024-09-21T07:27:11.419565+0000","last_became_peered":"2024-09-21T07:27:11.419565+0000","last_unstale":"2024-09-21T07:27:13.157555+0000","last_undegraded":"2024-09-21T07:27:13.157555+0000","last_fullsized":"2024-09-21T07:27:13.157555+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:32.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156709+0000","last_change":"2024-09-21T07:27:10.391324+0000","last_active":"2024-09-21T07:27:13.156709+0000","last_peered":"2024-09-21T07:27:13.156709+0000","last_clean":"2024-09-21T07:27:13.156709+0000","last_became_active":"2024-09-21T07:27:10.390663+0000","last_became_peered":"2024-09-21T07:27:10.390663+0000","last_unstale":"2024-09-21T07:27:13.156709+0000","last_undegraded":"2024-09-21T07:27:13.156709+0000","last_fullsized":"2024-09-21T07:27:13.156709+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:48.062789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156631+0000","last_change":"2024-09-21T07:27:11.406879+0000","last_active":"2024-09-21T07:27:13.156631+0000","last_peered":"2024-09-21T07:27:13.156631+0000","last_clean":"2024-09-21T07:27:13.156631+0000","last_became_active":"2024-09-21T07:27:11.406542+0000","last_became_peered":"2024-09-21T07:27:11.406542+0000","last_unstale":"2024-09-21T07:27:13.156631+0000","last_undegraded":"2024-09-21T07:27:13.156631+0000","last_fullsized":"2024-09-21T07:27:13.156631+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157695+0000","last_change":"2024-09-21T07:27:10.396250+0000","last_active":"2024-09-21T07:27:13.157695+0000","last_peered":"2024-09-21T07:27:13.157695+0000","last_clean":"2024-09-21T07:27:13.157695+0000","last_became_active":"2024-09-21T07:27:10.395234+0000","last_became_peered":"2024-09-21T07:27:10.395234+0000","last_unstale":"2024-09-21T07:27:13.157695+0000","last_undegraded":"2024-09-21T07:27:13.157695+0000","last_fullsized":"2024-09-21T07:27:13.157695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:49.882577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418595+0000","last_change":"2024-09-21T07:27:11.404215+0000","last_active":"2024-09-21T07:27:11.418595+0000","last_peered":"2024-09-21T07:27:11.418595+0000","last_clean":"2024-09-21T07:27:11.418595+0000","last_became_active":"2024-09-21T07:27:11.403888+0000","last_became_peered":"2024-09-21T07:27:11.403888+0000","last_unstale":"2024-09-21T07:27:11.418595+0000","last_undegraded":"2024-09-21T07:27:11.418595+0000","last_fullsized":"2024-09-21T07:27:11.418595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:43.851513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156601+0000","last_change":"2024-09-21T07:27:10.392527+0000","last_active":"2024-09-21T07:27:13.156601+0000","last_peered":"2024-09-21T07:27:13.156601+0000","last_clean":"2024-09-21T07:27:13.156601+0000","last_became_active":"2024-09-21T07:27:10.392193+0000","last_became_peered":"2024-09-21T07:27:10.392193+0000","last_unstale":"2024-09-21T07:27:13.156601+0000","last_undegraded":"2024-09-21T07:27:13.156601+0000","last_fullsized":"2024-09-21T07:27:13.156601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:39.389464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156508+0000","last_change":"2024-09-21T07:27:11.397331+0000","last_active":"2024-09-21T07:27:13.156508+0000","last_peered":"2024-09-21T07:27:13.156508+0000","last_clean":"2024-09-21T07:27:13.156508+0000","last_became_active":"2024-09-21T07:27:11.396883+0000","last_became_peered":"2024-09-21T07:27:11.396883+0000","last_unstale":"2024-09-21T07:27:13.156508+0000","last_undegraded":"2024-09-21T07:27:13.156508+0000","last_fullsized":"2024-09-21T07:27:13.156508+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409018+0000","last_change":"2024-09-21T07:27:10.395602+0000","last_active":"2024-09-21T07:27:12.409018+0000","last_peered":"2024-09-21T07:27:12.409018+0000","last_clean":"2024-09-21T07:27:12.409018+0000","last_became_active":"2024-09-21T07:27:10.395288+0000","last_became_peered":"2024-09-21T07:27:10.395288+0000","last_unstale":"2024-09-21T07:27:12.409018+0000","last_undegraded":"2024-09-21T07:27:12.409018+0000","last_fullsized":"2024-09-21T07:27:12.409018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:56.243273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409098+0000","last_change":"2024-09-21T07:27:11.416365+0000","last_active":"2024-09-21T07:27:12.409098+0000","last_peered":"2024-09-21T07:27:12.409098+0000","last_clean":"2024-09-21T07:27:12.409098+0000","last_became_active":"2024-09-21T07:27:11.415537+0000","last_became_peered":"2024-09-21T07:27:11.415537+0000","last_unstale":"2024-09-21T07:27:12.409098+0000","last_undegraded":"2024-09-21T07:27:12.409098+0000","last_fullsized":"2024-09-21T07:27:12.409098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:25.786724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393399+0000","last_change":"2024-09-21T07:27:10.396727+0000","last_active":"2024-09-21T07:27:12.393399+0000","last_peered":"2024-09-21T07:27:12.393399+0000","last_clean":"2024-09-21T07:27:12.393399+0000","last_became_active":"2024-09-21T07:27:10.396337+0000","last_became_peered":"2024-09-21T07:27:10.396337+0000","last_unstale":"2024-09-21T07:27:12.393399+0000","last_undegraded":"2024-09-21T07:27:12.393399+0000","last_fullsized":"2024-09-21T07:27:12.393399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:42.372953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156627+0000","last_change":"2024-09-21T07:27:11.419680+0000","last_active":"2024-09-21T07:27:13.156627+0000","last_peered":"2024-09-21T07:27:13.156627+0000","last_clean":"2024-09-21T07:27:13.156627+0000","last_became_active":"2024-09-21T07:27:11.419364+0000","last_became_peered":"2024-09-21T07:27:11.419364+0000","last_unstale":"2024-09-21T07:27:13.156627+0000","last_undegraded":"2024-09-21T07:27:13.156627+0000","last_fullsized":"2024-09-21T07:27:13.156627+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:43.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157075+0000","last_change":"2024-09-21T07:27:10.398502+0000","last_active":"2024-09-21T07:27:13.157075+0000","last_peered":"2024-09-21T07:27:13.157075+0000","last_clean":"2024-09-21T07:27:13.157075+0000","last_became_active":"2024-09-21T07:27:10.398173+0000","last_became_peered":"2024-09-21T07:27:10.398173+0000","last_unstale":"2024-09-21T07:27:13.157075+0000","last_undegraded":"2024-09-21T07:27:13.157075+0000","last_fullsized":"2024-09-21T07:27:13.157075+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:43.323314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418635+0000","last_change":"2024-09-21T07:27:11.403570+0000","last_active":"2024-09-21T07:27:11.418635+0000","last_peered":"2024-09-21T07:27:11.418635+0000","last_clean":"2024-09-21T07:27:11.418635+0000","last_became_active":"2024-09-21T07:27:11.403259+0000","last_became_peered":"2024-09-21T07:27:11.403259+0000","last_unstale":"2024-09-21T07:27:11.418635+0000","last_undegraded":"2024-09-21T07:27:11.418635+0000","last_fullsized":"2024-09-21T07:27:11.418635+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:41.232891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156196+0000","last_change":"2024-09-21T07:27:10.387763+0000","last_active":"2024-09-21T07:27:13.156196+0000","last_peered":"2024-09-21T07:27:13.156196+0000","last_clean":"2024-09-21T07:27:13.156196+0000","last_became_active":"2024-09-21T07:27:10.386289+0000","last_became_peered":"2024-09-21T07:27:10.386289+0000","last_unstale":"2024-09-21T07:27:13.156196+0000","last_undegraded":"2024-09-21T07:27:13.156196+0000","last_fullsized":"2024-09-21T07:27:13.156196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:57.843041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406560+0000","last_change":"2024-09-21T07:27:11.408508+0000","last_active":"2024-09-21T07:27:12.406560+0000","last_peered":"2024-09-21T07:27:12.406560+0000","last_clean":"2024-09-21T07:27:12.406560+0000","last_became_active":"2024-09-21T07:27:11.408199+0000","last_became_peered":"2024-09-21T07:27:11.408199+0000","last_unstale":"2024-09-21T07:27:12.406560+0000","last_undegraded":"2024-09-21T07:27:12.406560+0000","last_fullsized":"2024-09-21T07:27:12.406560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:28.179893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.388759+0000","last_change":"2024-09-21T07:27:10.388613+0000","last_active":"2024-09-21T07:27:11.388759+0000","last_peered":"2024-09-21T07:27:11.388759+0000","last_clean":"2024-09-21T07:27:11.388759+0000","last_became_active":"2024-09-21T07:27:10.388295+0000","last_became_peered":"2024-09-21T07:27:10.388295+0000","last_unstale":"2024-09-21T07:27:11.388759+0000","last_undegraded":"2024-09-21T07:27:11.388759+0000","last_fullsized":"2024-09-21T07:27:11.388759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:52.685938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394547+0000","last_change":"2024-09-21T07:27:11.410517+0000","last_active":"2024-09-21T07:27:12.394547+0000","last_peered":"2024-09-21T07:27:12.394547+0000","last_clean":"2024-09-21T07:27:12.394547+0000","last_became_active":"2024-09-21T07:27:11.410184+0000","last_became_peered":"2024-09-21T07:27:11.410184+0000","last_unstale":"2024-09-21T07:27:12.394547+0000","last_undegraded":"2024-09-21T07:27:12.394547+0000","last_fullsized":"2024-09-21T07:27:12.394547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:24.360375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409433+0000","last_change":"2024-09-21T07:27:10.395921+0000","last_active":"2024-09-21T07:27:12.409433+0000","last_peered":"2024-09-21T07:27:12.409433+0000","last_clean":"2024-09-21T07:27:12.409433+0000","last_became_active":"2024-09-21T07:27:10.394941+0000","last_became_peered":"2024-09-21T07:27:10.394941+0000","last_unstale":"2024-09-21T07:27:12.409433+0000","last_undegraded":"2024-09-21T07:27:12.409433+0000","last_fullsized":"2024-09-21T07:27:12.409433+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:32.180578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157104+0000","last_change":"2024-09-21T07:27:11.419437+0000","last_active":"2024-09-21T07:27:13.157104+0000","last_peered":"2024-09-21T07:27:13.157104+0000","last_clean":"2024-09-21T07:27:13.157104+0000","last_became_active":"2024-09-21T07:27:11.419125+0000","last_became_peered":"2024-09-21T07:27:11.419125+0000","last_unstale":"2024-09-21T07:27:13.157104+0000","last_undegraded":"2024-09-21T07:27:13.157104+0000","last_fullsized":"2024-09-21T07:27:13.157104+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:40.507567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409430+0000","last_change":"2024-09-21T07:27:10.395815+0000","last_active":"2024-09-21T07:27:12.409430+0000","last_peered":"2024-09-21T07:27:12.409430+0000","last_clean":"2024-09-21T07:27:12.409430+0000","last_became_active":"2024-09-21T07:27:10.395395+0000","last_became_peered":"2024-09-21T07:27:10.395395+0000","last_unstale":"2024-09-21T07:27:12.409430+0000","last_undegraded":"2024-09-21T07:27:12.409430+0000","last_fullsized":"2024-09-21T07:27:12.409430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.237028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156673+0000","last_change":"2024-09-21T07:27:11.398272+0000","last_active":"2024-09-21T07:27:13.156673+0000","last_peered":"2024-09-21T07:27:13.156673+0000","last_clean":"2024-09-21T07:27:13.156673+0000","last_became_active":"2024-09-21T07:27:11.397821+0000","last_became_peered":"2024-09-21T07:27:11.397821+0000","last_unstale":"2024-09-21T07:27:13.156673+0000","last_undegraded":"2024-09-21T07:27:13.156673+0000","last_fullsized":"2024-09-21T07:27:13.156673+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:53.111009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.428317+0000","last_change":"2024-09-21T07:27:10.391438+0000","last_active":"2024-09-21T07:27:11.428317+0000","last_peered":"2024-09-21T07:27:11.428317+0000","last_clean":"2024-09-21T07:27:11.428317+0000","last_became_active":"2024-09-21T07:27:10.391138+0000","last_became_peered":"2024-09-21T07:27:10.391138+0000","last_unstale":"2024-09-21T07:27:11.428317+0000","last_undegraded":"2024-09-21T07:27:11.428317+0000","last_fullsized":"2024-09-21T07:27:11.428317+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:41.148308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156775+0000","last_change":"2024-09-21T07:27:11.416980+0000","last_active":"2024-09-21T07:27:13.156775+0000","last_peered":"2024-09-21T07:27:13.156775+0000","last_clean":"2024-09-21T07:27:13.156775+0000","last_became_active":"2024-09-21T07:27:11.416627+0000","last_became_peered":"2024-09-21T07:27:11.416627+0000","last_unstale":"2024-09-21T07:27:13.156775+0000","last_undegraded":"2024-09-21T07:27:13.156775+0000","last_fullsized":"2024-09-21T07:27:13.156775+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420702+0000","last_change":"2024-09-21T07:27:10.392664+0000","last_active":"2024-09-21T07:27:11.420702+0000","last_peered":"2024-09-21T07:27:11.420702+0000","last_clean":"2024-09-21T07:27:11.420702+0000","last_became_active":"2024-09-21T07:27:10.392208+0000","last_became_peered":"2024-09-21T07:27:10.392208+0000","last_unstale":"2024-09-21T07:27:11.420702+0000","last_undegraded":"2024-09-21T07:27:11.420702+0000","last_fullsized":"2024-09-21T07:27:11.420702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:53.537965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.405059+0000","last_change":"2024-09-21T07:27:11.400024+0000","last_active":"2024-09-21T07:27:11.405059+0000","last_peered":"2024-09-21T07:27:11.405059+0000","last_clean":"2024-09-21T07:27:11.405059+0000","last_became_active":"2024-09-21T07:27:11.399714+0000","last_became_peered":"2024-09-21T07:27:11.399714+0000","last_unstale":"2024-09-21T07:27:11.405059+0000","last_undegraded":"2024-09-21T07:27:11.405059+0000","last_fullsized":"2024-09-21T07:27:11.405059+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:58.985253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157213+0000","last_change":"2024-09-21T07:27:10.397596+0000","last_active":"2024-09-21T07:27:13.157213+0000","last_peered":"2024-09-21T07:27:13.157213+0000","last_clean":"2024-09-21T07:27:13.157213+0000","last_became_active":"2024-09-21T07:27:10.397245+0000","last_became_peered":"2024-09-21T07:27:10.397245+0000","last_unstale":"2024-09-21T07:27:13.157213+0000","last_undegraded":"2024-09-21T07:27:13.157213+0000","last_fullsized":"2024-09-21T07:27:13.157213+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:48.062789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393142+0000","last_change":"2024-09-21T07:27:11.409267+0000","last_active":"2024-09-21T07:27:12.393142+0000","last_peered":"2024-09-21T07:27:12.393142+0000","last_clean":"2024-09-21T07:27:12.393142+0000","last_became_active":"2024-09-21T07:27:11.408350+0000","last_became_peered":"2024-09-21T07:27:11.408350+0000","last_unstale":"2024-09-21T07:27:12.393142+0000","last_undegraded":"2024-09-21T07:27:12.393142+0000","last_fullsized":"2024-09-21T07:27:12.393142+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:53.111009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.392828+0000","last_change":"2024-09-21T07:27:10.386578+0000","last_active":"2024-09-21T07:27:12.392828+0000","last_peered":"2024-09-21T07:27:12.392828+0000","last_clean":"2024-09-21T07:27:12.392828+0000","last_became_active":"2024-09-21T07:27:10.386265+0000","last_became_peered":"2024-09-21T07:27:10.386265+0000","last_unstale":"2024-09-21T07:27:12.392828+0000","last_undegraded":"2024-09-21T07:27:12.392828+0000","last_fullsized":"2024-09-21T07:27:12.392828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:32.393599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.407427+0000","last_change":"2024-09-21T07:27:11.400834+0000","last_active":"2024-09-21T07:27:11.407427+0000","last_peered":"2024-09-21T07:27:11.407427+0000","last_clean":"2024-09-21T07:27:11.407427+0000","last_became_active":"2024-09-21T07:27:11.400546+0000","last_became_peered":"2024-09-21T07:27:11.400546+0000","last_unstale":"2024-09-21T07:27:11.407427+0000","last_undegraded":"2024-09-21T07:27:11.407427+0000","last_fullsized":"2024-09-21T07:27:11.407427+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:30.482260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393238+0000","last_change":"2024-09-21T07:27:10.397744+0000","last_active":"2024-09-21T07:27:12.393238+0000","last_peered":"2024-09-21T07:27:12.393238+0000","last_clean":"2024-09-21T07:27:12.393238+0000","last_became_active":"2024-09-21T07:27:10.397379+0000","last_became_peered":"2024-09-21T07:27:10.397379+0000","last_unstale":"2024-09-21T07:27:12.393238+0000","last_undegraded":"2024-09-21T07:27:12.393238+0000","last_fullsized":"2024-09-21T07:27:12.393238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:57.843041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420282+0000","last_change":"2024-09-21T07:27:11.407006+0000","last_active":"2024-09-21T07:27:11.420282+0000","last_peered":"2024-09-21T07:27:11.420282+0000","last_clean":"2024-09-21T07:27:11.420282+0000","last_became_active":"2024-09-21T07:27:11.406669+0000","last_became_peered":"2024-09-21T07:27:11.406669+0000","last_unstale":"2024-09-21T07:27:11.420282+0000","last_undegraded":"2024-09-21T07:27:11.420282+0000","last_fullsized":"2024-09-21T07:27:11.420282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:19.675326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409480+0000","last_change":"2024-09-21T07:27:10.397819+0000","last_active":"2024-09-21T07:27:12.409480+0000","last_peered":"2024-09-21T07:27:12.409480+0000","last_clean":"2024-09-21T07:27:12.409480+0000","last_became_active":"2024-09-21T07:27:10.397418+0000","last_became_peered":"2024-09-21T07:27:10.397418+0000","last_unstale":"2024-09-21T07:27:12.409480+0000","last_undegraded":"2024-09-21T07:27:12.409480+0000","last_fullsized":"2024-09-21T07:27:12.409480+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:30.816349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409362+0000","last_change":"2024-09-21T07:27:11.417488+0000","last_active":"2024-09-21T07:27:12.409362+0000","last_peered":"2024-09-21T07:27:12.409362+0000","last_clean":"2024-09-21T07:27:12.409362+0000","last_became_active":"2024-09-21T07:27:11.416782+0000","last_became_peered":"2024-09-21T07:27:11.416782+0000","last_unstale":"2024-09-21T07:27:12.409362+0000","last_undegraded":"2024-09-21T07:27:12.409362+0000","last_fullsized":"2024-09-21T07:27:12.409362+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:41.958678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156768+0000","last_change":"2024-09-21T07:27:10.395859+0000","last_active":"2024-09-21T07:27:13.156768+0000","last_peered":"2024-09-21T07:27:13.156768+0000","last_clean":"2024-09-21T07:27:13.156768+0000","last_became_active":"2024-09-21T07:27:10.395549+0000","last_became_peered":"2024-09-21T07:27:10.395549+0000","last_unstale":"2024-09-21T07:27:13.156768+0000","last_undegraded":"2024-09-21T07:27:13.156768+0000","last_fullsized":"2024-09-21T07:27:13.156768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:56.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156600+0000","last_change":"2024-09-21T07:27:11.399404+0000","last_active":"2024-09-21T07:27:13.156600+0000","last_peered":"2024-09-21T07:27:13.156600+0000","last_clean":"2024-09-21T07:27:13.156600+0000","last_became_active":"2024-09-21T07:27:11.398941+0000","last_became_peered":"2024-09-21T07:27:11.398941+0000","last_unstale":"2024-09-21T07:27:13.156600+0000","last_undegraded":"2024-09-21T07:27:13.156600+0000","last_fullsized":"2024-09-21T07:27:13.156600+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:14.516851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406597+0000","last_change":"2024-09-21T07:27:10.396054+0000","last_active":"2024-09-21T07:27:12.406597+0000","last_peered":"2024-09-21T07:27:12.406597+0000","last_clean":"2024-09-21T07:27:12.406597+0000","last_became_active":"2024-09-21T07:27:10.395716+0000","last_became_peered":"2024-09-21T07:27:10.395716+0000","last_unstale":"2024-09-21T07:27:12.406597+0000","last_undegraded":"2024-09-21T07:27:12.406597+0000","last_fullsized":"2024-09-21T07:27:12.406597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:25.025889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406737+0000","last_change":"2024-09-21T07:27:11.397987+0000","last_active":"2024-09-21T07:27:12.406737+0000","last_peered":"2024-09-21T07:27:12.406737+0000","last_clean":"2024-09-21T07:27:12.406737+0000","last_became_active":"2024-09-21T07:27:11.397691+0000","last_became_peered":"2024-09-21T07:27:11.397691+0000","last_unstale":"2024-09-21T07:27:12.406737+0000","last_undegraded":"2024-09-21T07:27:12.406737+0000","last_fullsized":"2024-09-21T07:27:12.406737+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:43.187380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157356+0000","last_change":"2024-09-21T07:27:10.398432+0000","last_active":"2024-09-21T07:27:13.157356+0000","last_peered":"2024-09-21T07:27:13.157356+0000","last_clean":"2024-09-21T07:27:13.157356+0000","last_became_active":"2024-09-21T07:27:10.397959+0000","last_became_peered":"2024-09-21T07:27:10.397959+0000","last_unstale":"2024-09-21T07:27:13.157356+0000","last_undegraded":"2024-09-21T07:27:13.157356+0000","last_fullsized":"2024-09-21T07:27:13.157356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:45.381411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393115+0000","last_change":"2024-09-21T07:27:11.414577+0000","last_active":"2024-09-21T07:27:12.393115+0000","last_peered":"2024-09-21T07:27:12.393115+0000","last_clean":"2024-09-21T07:27:12.393115+0000","last_became_active":"2024-09-21T07:27:11.414248+0000","last_became_peered":"2024-09-21T07:27:11.414248+0000","last_unstale":"2024-09-21T07:27:12.393115+0000","last_undegraded":"2024-09-21T07:27:12.393115+0000","last_fullsized":"2024-09-21T07:27:12.393115+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:00.795137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406452+0000","last_change":"2024-09-21T07:27:10.395227+0000","last_active":"2024-09-21T07:27:12.406452+0000","last_peered":"2024-09-21T07:27:12.406452+0000","last_clean":"2024-09-21T07:27:12.406452+0000","last_became_active":"2024-09-21T07:27:10.394862+0000","last_became_peered":"2024-09-21T07:27:10.394862+0000","last_unstale":"2024-09-21T07:27:12.406452+0000","last_undegraded":"2024-09-21T07:27:12.406452+0000","last_fullsized":"2024-09-21T07:27:12.406452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:39.418324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156545+0000","last_change":"2024-09-21T07:27:11.402672+0000","last_active":"2024-09-21T07:27:13.156545+0000","last_peered":"2024-09-21T07:27:13.156545+0000","last_clean":"2024-09-21T07:27:13.156545+0000","last_became_active":"2024-09-21T07:27:11.402364+0000","last_became_peered":"2024-09-21T07:27:11.402364+0000","last_unstale":"2024-09-21T07:27:13.156545+0000","last_undegraded":"2024-09-21T07:27:13.156545+0000","last_fullsized":"2024-09-21T07:27:13.156545+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:53.609448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406374+0000","last_change":"2024-09-21T07:27:10.396828+0000","last_active":"2024-09-21T07:27:12.406374+0000","last_peered":"2024-09-21T07:27:12.406374+0000","last_clean":"2024-09-21T07:27:12.406374+0000","last_became_active":"2024-09-21T07:27:10.396238+0000","last_became_peered":"2024-09-21T07:27:10.396238+0000","last_unstale":"2024-09-21T07:27:12.406374+0000","last_undegraded":"2024-09-21T07:27:12.406374+0000","last_fullsized":"2024-09-21T07:27:12.406374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:28.076759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408952+0000","last_change":"2024-09-21T07:27:11.416310+0000","last_active":"2024-09-21T07:27:12.408952+0000","last_peered":"2024-09-21T07:27:12.408952+0000","last_clean":"2024-09-21T07:27:12.408952+0000","last_became_active":"2024-09-21T07:27:11.415973+0000","last_became_peered":"2024-09-21T07:27:11.415973+0000","last_unstale":"2024-09-21T07:27:12.408952+0000","last_undegraded":"2024-09-21T07:27:12.408952+0000","last_fullsized":"2024-09-21T07:27:12.408952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:57.999550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393539+0000","last_change":"2024-09-21T07:27:10.392540+0000","last_active":"2024-09-21T07:27:12.393539+0000","last_peered":"2024-09-21T07:27:12.393539+0000","last_clean":"2024-09-21T07:27:12.393539+0000","last_became_active":"2024-09-21T07:27:10.392144+0000","last_became_peered":"2024-09-21T07:27:10.392144+0000","last_unstale":"2024-09-21T07:27:12.393539+0000","last_undegraded":"2024-09-21T07:27:12.393539+0000","last_fullsized":"2024-09-21T07:27:12.393539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:39.389464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418664+0000","last_change":"2024-09-21T07:27:11.404678+0000","last_active":"2024-09-21T07:27:11.418664+0000","last_peered":"2024-09-21T07:27:11.418664+0000","last_clean":"2024-09-21T07:27:11.418664+0000","last_became_active":"2024-09-21T07:27:11.404385+0000","last_became_peered":"2024-09-21T07:27:11.404385+0000","last_unstale":"2024-09-21T07:27:11.418664+0000","last_undegraded":"2024-09-21T07:27:11.418664+0000","last_fullsized":"2024-09-21T07:27:11.418664+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:24.952994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406289+0000","last_change":"2024-09-21T07:27:10.394546+0000","last_active":"2024-09-21T07:27:12.406289+0000","last_peered":"2024-09-21T07:27:12.406289+0000","last_clean":"2024-09-21T07:27:12.406289+0000","last_became_active":"2024-09-21T07:27:10.394098+0000","last_became_peered":"2024-09-21T07:27:10.394098+0000","last_unstale":"2024-09-21T07:27:12.406289+0000","last_undegraded":"2024-09-21T07:27:12.406289+0000","last_fullsized":"2024-09-21T07:27:12.406289+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:46.231370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157074+0000","last_change":"2024-09-21T07:27:11.419603+0000","last_active":"2024-09-21T07:27:13.157074+0000","last_peered":"2024-09-21T07:27:13.157074+0000","last_clean":"2024-09-21T07:27:13.157074+0000","last_became_active":"2024-09-21T07:27:11.419312+0000","last_became_peered":"2024-09-21T07:27:11.419312+0000","last_unstale":"2024-09-21T07:27:13.157074+0000","last_undegraded":"2024-09-21T07:27:13.157074+0000","last_fullsized":"2024-09-21T07:27:13.157074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409547+0000","last_change":"2024-09-21T07:27:10.393397+0000","last_active":"2024-09-21T07:27:12.409547+0000","last_peered":"2024-09-21T07:27:12.409547+0000","last_clean":"2024-09-21T07:27:12.409547+0000","last_became_active":"2024-09-21T07:27:10.393068+0000","last_became_peered":"2024-09-21T07:27:10.393068+0000","last_unstale":"2024-09-21T07:27:12.409547+0000","last_undegraded":"2024-09-21T07:27:12.409547+0000","last_fullsized":"2024-09-21T07:27:12.409547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:19.651643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394534+0000","last_change":"2024-09-21T07:27:11.411931+0000","last_active":"2024-09-21T07:27:12.394534+0000","last_peered":"2024-09-21T07:27:12.394534+0000","last_clean":"2024-09-21T07:27:12.394534+0000","last_became_active":"2024-09-21T07:27:11.411616+0000","last_became_peered":"2024-09-21T07:27:11.411616+0000","last_unstale":"2024-09-21T07:27:12.394534+0000","last_undegraded":"2024-09-21T07:27:12.394534+0000","last_fullsized":"2024-09-21T07:27:12.394534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:03.851132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.407918+0000","last_change":"2024-09-21T07:27:10.398924+0000","last_active":"2024-09-21T07:27:12.407918+0000","last_peered":"2024-09-21T07:27:12.407918+0000","last_clean":"2024-09-21T07:27:12.407918+0000","last_became_active":"2024-09-21T07:27:10.398676+0000","last_became_peered":"2024-09-21T07:27:10.398676+0000","last_unstale":"2024-09-21T07:27:12.407918+0000","last_undegraded":"2024-09-21T07:27:12.407918+0000","last_fullsized":"2024-09-21T07:27:12.407918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:18.062836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.407902+0000","last_change":"2024-09-21T07:27:11.402449+0000","last_active":"2024-09-21T07:27:11.407902+0000","last_peered":"2024-09-21T07:27:11.407902+0000","last_clean":"2024-09-21T07:27:11.407902+0000","last_became_active":"2024-09-21T07:27:11.402178+0000","last_became_peered":"2024-09-21T07:27:11.402178+0000","last_unstale":"2024-09-21T07:27:11.407902+0000","last_undegraded":"2024-09-21T07:27:11.407902+0000","last_fullsized":"2024-09-21T07:27:11.407902+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:31.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156724+0000","last_change":"2024-09-21T07:27:10.390528+0000","last_active":"2024-09-21T07:27:13.156724+0000","last_peered":"2024-09-21T07:27:13.156724+0000","last_clean":"2024-09-21T07:27:13.156724+0000","last_became_active":"2024-09-21T07:27:10.390230+0000","last_became_peered":"2024-09-21T07:27:10.390230+0000","last_unstale":"2024-09-21T07:27:13.156724+0000","last_undegraded":"2024-09-21T07:27:13.156724+0000","last_fullsized":"2024-09-21T07:27:13.156724+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.319182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156834+0000","last_change":"2024-09-21T07:27:11.406593+0000","last_active":"2024-09-21T07:27:13.156834+0000","last_peered":"2024-09-21T07:27:13.156834+0000","last_clean":"2024-09-21T07:27:13.156834+0000","last_became_active":"2024-09-21T07:27:11.406229+0000","last_became_peered":"2024-09-21T07:27:11.406229+0000","last_unstale":"2024-09-21T07:27:13.156834+0000","last_undegraded":"2024-09-21T07:27:13.156834+0000","last_fullsized":"2024-09-21T07:27:13.156834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.803577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157045+0000","last_change":"2024-09-21T07:27:10.390195+0000","last_active":"2024-09-21T07:27:13.157045+0000","last_peered":"2024-09-21T07:27:13.157045+0000","last_clean":"2024-09-21T07:27:13.157045+0000","last_became_active":"2024-09-21T07:27:10.389881+0000","last_became_peered":"2024-09-21T07:27:10.389881+0000","last_unstale":"2024-09-21T07:27:13.157045+0000","last_undegraded":"2024-09-21T07:27:13.157045+0000","last_fullsized":"2024-09-21T07:27:13.157045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:45.381411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157341+0000","last_change":"2024-09-21T07:27:11.405867+0000","last_active":"2024-09-21T07:27:13.157341+0000","last_peered":"2024-09-21T07:27:13.157341+0000","last_clean":"2024-09-21T07:27:13.157341+0000","last_became_active":"2024-09-21T07:27:11.405535+0000","last_became_peered":"2024-09-21T07:27:11.405535+0000","last_unstale":"2024-09-21T07:27:13.157341+0000","last_undegraded":"2024-09-21T07:27:13.157341+0000","last_fullsized":"2024-09-21T07:27:13.157341+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:21.318618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393739+0000","last_change":"2024-09-21T07:27:10.391250+0000","last_active":"2024-09-21T07:27:12.393739+0000","last_peered":"2024-09-21T07:27:12.393739+0000","last_clean":"2024-09-21T07:27:12.393739+0000","last_became_active":"2024-09-21T07:27:10.390890+0000","last_became_peered":"2024-09-21T07:27:10.390890+0000","last_unstale":"2024-09-21T07:27:12.393739+0000","last_undegraded":"2024-09-21T07:27:12.393739+0000","last_fullsized":"2024-09-21T07:27:12.393739+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:42.181705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156373+0000","last_change":"2024-09-21T07:27:11.406895+0000","last_active":"2024-09-21T07:27:13.156373+0000","last_peered":"2024-09-21T07:27:13.156373+0000","last_clean":"2024-09-21T07:27:13.156373+0000","last_became_active":"2024-09-21T07:27:11.406525+0000","last_became_peered":"2024-09-21T07:27:11.406525+0000","last_unstale":"2024-09-21T07:27:13.156373+0000","last_undegraded":"2024-09-21T07:27:13.156373+0000","last_fullsized":"2024-09-21T07:27:13.156373+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:21.318618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156179+0000","last_change":"2024-09-21T07:27:10.387436+0000","last_active":"2024-09-21T07:27:13.156179+0000","last_peered":"2024-09-21T07:27:13.156179+0000","last_clean":"2024-09-21T07:27:13.156179+0000","last_became_active":"2024-09-21T07:27:10.386526+0000","last_became_peered":"2024-09-21T07:27:10.386526+0000","last_unstale":"2024-09-21T07:27:13.156179+0000","last_undegraded":"2024-09-21T07:27:13.156179+0000","last_fullsized":"2024-09-21T07:27:13.156179+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:42.372953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394310+0000","last_change":"2024-09-21T07:27:11.411274+0000","last_active":"2024-09-21T07:27:12.394310+0000","last_peered":"2024-09-21T07:27:12.394310+0000","last_clean":"2024-09-21T07:27:12.394310+0000","last_became_active":"2024-09-21T07:27:11.410821+0000","last_became_peered":"2024-09-21T07:27:11.410821+0000","last_unstale":"2024-09-21T07:27:12.394310+0000","last_undegraded":"2024-09-21T07:27:12.394310+0000","last_fullsized":"2024-09-21T07:27:12.394310+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:39.773957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.387814+0000","last_change":"2024-09-21T07:27:10.387628+0000","last_active":"2024-09-21T07:27:11.387814+0000","last_peered":"2024-09-21T07:27:11.387814+0000","last_clean":"2024-09-21T07:27:11.387814+0000","last_became_active":"2024-09-21T07:27:10.387344+0000","last_became_peered":"2024-09-21T07:27:10.387344+0000","last_unstale":"2024-09-21T07:27:11.387814+0000","last_undegraded":"2024-09-21T07:27:11.387814+0000","last_fullsized":"2024-09-21T07:27:11.387814+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:30.294147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409446+0000","last_change":"2024-09-21T07:27:11.422387+0000","last_active":"2024-09-21T07:27:12.409446+0000","last_peered":"2024-09-21T07:27:12.409446+0000","last_clean":"2024-09-21T07:27:12.409446+0000","last_became_active":"2024-09-21T07:27:11.422059+0000","last_became_peered":"2024-09-21T07:27:11.422059+0000","last_unstale":"2024-09-21T07:27:12.409446+0000","last_undegraded":"2024-09-21T07:27:12.409446+0000","last_fullsized":"2024-09-21T07:27:12.409446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:16.559852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394273+0000","last_change":"2024-09-21T07:27:10.392863+0000","last_active":"2024-09-21T07:27:12.394273+0000","last_peered":"2024-09-21T07:27:12.394273+0000","last_clean":"2024-09-21T07:27:12.394273+0000","last_became_active":"2024-09-21T07:27:10.392554+0000","last_became_peered":"2024-09-21T07:27:10.392554+0000","last_unstale":"2024-09-21T07:27:12.394273+0000","last_undegraded":"2024-09-21T07:27:12.394273+0000","last_fullsized":"2024-09-21T07:27:12.394273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:56.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":148,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393182+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:12.393182+0000","last_peered":"2024-09-21T07:27:12.393182+0000","last_clean":"2024-09-21T07:27:12.393182+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:12.393182+0000","last_undegraded":"2024-09-21T07:27:12.393182+0000","last_fullsized":"2024-09-21T07:27:12.393182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":795717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":204249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":203779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":475136,"data_stored":204271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":203737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":794069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":196975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:15.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.565+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/1718062838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 msgr2=0x7f7e700052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.565+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/1718062838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e700052b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e840c93b0 tx=0x7f7e70005830 comp rx=0 tx=0).stop 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.565+0000 7f7e89957640 1 -- 172.21.15.105:0/1718062838 reap_dead start 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.566+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/1718062838 shutdown_connections 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.566+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/1718062838 >> 172.21.15.105:0/1718062838 conn(0x7f7e840cedb0 msgr2=0x7f7e840cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.566+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/1718062838 shutdown_connections 2024-09-21T07:27:15.568 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.566+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/1718062838 wait complete. 2024-09-21T07:27:15.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.567+0000 7f7e8b3ba640 1 Processor -- start 2024-09-21T07:27:15.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.567+0000 7f7e8b3ba640 1 -- start start 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e8415e010 0x7f7e8415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e8415a760 0x7f7e8415ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7e840cebe0 con 0x7f7e70004e60 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e840cea60 con 0x7f7e8415e010 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7e840d0310 con 0x7f7e8415a760 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58336/0 (socket says 172.21.15.105:58336) 2024-09-21T07:27:15.572 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 -- 172.21.15.105:0/2081053223 learned_addr learned my addr 172.21.15.105:0/2081053223 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e88955640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e8415e010 0x7f7e8415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89957640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e8415a760 0x7f7e8415ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 -- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e8415a760 msgr2=0x7f7e8415ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e8415a760 0x7f7e8415ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 -- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e8415e010 msgr2=0x7f7e8415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e8415e010 0x7f7e8415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 -- 172.21.15.105:0/2081053223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8415b0b0 con 0x7f7e70004e60 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e89156640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e80003ec0 tx=0x7f7e80001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80015020 con 0x7f7e70004e60 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e80011840 con 0x7f7e70004e60 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841f3a90 con 0x7f7e70004e60 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.569+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80011c50 con 0x7f7e70004e60 2024-09-21T07:27:15.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.570+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e841f4050 con 0x7f7e70004e60 2024-09-21T07:27:15.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.571+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e840727f0 con 0x7f7e70004e60 2024-09-21T07:27:15.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.571+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7e80018020 con 0x7f7e70004e60 2024-09-21T07:27:15.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.572+0000 7f7e79ffb640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 0x7f7e6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.572+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7e8009b970 con 0x7f7e70004e60 2024-09-21T07:27:15.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.572+0000 7f7e88955640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 0x7f7e6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.572+0000 7f7e88955640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 0x7f7e6807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e840cc340 tx=0x7f7e7400b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.578 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.576+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e8001f020 con 0x7f7e70004e60 2024-09-21T07:27:15.749 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.747+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e841f3810 con 0x7f7e68078b90 2024-09-21T07:27:15.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.752+0000 7f7e79ffb640 1 -- 172.21.15.105:0/2081053223 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f7e7400b7d0 con 0x7f7e68078b90 2024-09-21T07:27:15.756 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:15.758 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 msgr2=0x7f7e6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 0x7f7e6807b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7e840cc340 tx=0x7f7e7400b260 comp rx=0 tx=0).stop 2024-09-21T07:27:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 msgr2=0x7f7e8409f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7e80003ec0 tx=0x7f7e80001630 comp rx=0 tx=0).stop 2024-09-21T07:27:15.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 shutdown_connections 2024-09-21T07:27:15.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7e68078b90 0x7f7e6807b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e8415e010 0x7f7e8415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e8415a760 0x7f7e8415ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 --2- 172.21.15.105:0/2081053223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7e70004e60 0x7f7e8409f7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 >> 172.21.15.105:0/2081053223 conn(0x7f7e840cedb0 msgr2=0x7f7e840af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 shutdown_connections 2024-09-21T07:27:15.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.755+0000 7f7e8b3ba640 1 -- 172.21.15.105:0/2081053223 wait complete. 2024-09-21T07:27:15.768 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-21T07:27:14.780401+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221084,"kb_used_data":4844,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753636,"statfs":{"total":773094113280,"available":772867723264,"internally_reserved":0,"allocated":4960256,"data_stored":2806554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.407869"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.405331+0000","last_change":"2024-09-21T07:27:11.400818+0000","last_active":"2024-09-21T07:27:11.405331+0000","last_peered":"2024-09-21T07:27:11.405331+0000","last_clean":"2024-09-21T07:27:11.405331+0000","last_became_active":"2024-09-21T07:27:11.400518+0000","last_became_peered":"2024-09-21T07:27:11.400518+0000","last_unstale":"2024-09-21T07:27:11.405331+0000","last_undegraded":"2024-09-21T07:27:11.405331+0000","last_fullsized":"2024-09-21T07:27:11.405331+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:35.470013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.404264+0000","last_change":"2024-09-21T07:27:11.399663+0000","last_active":"2024-09-21T07:27:11.404264+0000","last_peered":"2024-09-21T07:27:11.404264+0000","last_clean":"2024-09-21T07:27:11.404264+0000","last_became_active":"2024-09-21T07:27:11.399345+0000","last_became_peered":"2024-09-21T07:27:11.399345+0000","last_unstale":"2024-09-21T07:27:11.404264+0000","last_undegraded":"2024-09-21T07:27:11.404264+0000","last_fullsized":"2024-09-21T07:27:11.404264+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:16.101562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156904+0000","last_change":"2024-09-21T07:27:11.420097+0000","last_active":"2024-09-21T07:27:13.156904+0000","last_peered":"2024-09-21T07:27:13.156904+0000","last_clean":"2024-09-21T07:27:13.156904+0000","last_became_active":"2024-09-21T07:27:11.419711+0000","last_became_peered":"2024-09-21T07:27:11.419711+0000","last_unstale":"2024-09-21T07:27:13.156904+0000","last_undegraded":"2024-09-21T07:27:13.156904+0000","last_fullsized":"2024-09-21T07:27:13.156904+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:57.840508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406514+0000","last_change":"2024-09-21T07:27:11.412520+0000","last_active":"2024-09-21T07:27:12.406514+0000","last_peered":"2024-09-21T07:27:12.406514+0000","last_clean":"2024-09-21T07:27:12.406514+0000","last_became_active":"2024-09-21T07:27:11.412196+0000","last_became_peered":"2024-09-21T07:27:11.412196+0000","last_unstale":"2024-09-21T07:27:12.406514+0000","last_undegraded":"2024-09-21T07:27:12.406514+0000","last_fullsized":"2024-09-21T07:27:12.406514+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:04.293157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406422+0000","last_change":"2024-09-21T07:27:11.412623+0000","last_active":"2024-09-21T07:27:12.406422+0000","last_peered":"2024-09-21T07:27:12.406422+0000","last_clean":"2024-09-21T07:27:12.406422+0000","last_became_active":"2024-09-21T07:27:11.412142+0000","last_became_peered":"2024-09-21T07:27:11.412142+0000","last_unstale":"2024-09-21T07:27:12.406422+0000","last_undegraded":"2024-09-21T07:27:12.406422+0000","last_fullsized":"2024-09-21T07:27:12.406422+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:54.215621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157363+0000","last_change":"2024-09-21T07:27:11.406389+0000","last_active":"2024-09-21T07:27:13.157363+0000","last_peered":"2024-09-21T07:27:13.157363+0000","last_clean":"2024-09-21T07:27:13.157363+0000","last_became_active":"2024-09-21T07:27:11.406082+0000","last_became_peered":"2024-09-21T07:27:11.406082+0000","last_unstale":"2024-09-21T07:27:13.157363+0000","last_undegraded":"2024-09-21T07:27:13.157363+0000","last_fullsized":"2024-09-21T07:27:13.157363+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:51.487806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408798+0000","last_change":"2024-09-21T07:27:11.423122+0000","last_active":"2024-09-21T07:27:12.408798+0000","last_peered":"2024-09-21T07:27:12.408798+0000","last_clean":"2024-09-21T07:27:12.408798+0000","last_became_active":"2024-09-21T07:27:11.421902+0000","last_became_peered":"2024-09-21T07:27:11.421902+0000","last_unstale":"2024-09-21T07:27:12.408798+0000","last_undegraded":"2024-09-21T07:27:12.408798+0000","last_fullsized":"2024-09-21T07:27:12.408798+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:13.731711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420887+0000","last_change":"2024-09-21T07:27:11.418117+0000","last_active":"2024-09-21T07:27:11.420887+0000","last_peered":"2024-09-21T07:27:11.420887+0000","last_clean":"2024-09-21T07:27:11.420887+0000","last_became_active":"2024-09-21T07:27:11.417686+0000","last_became_peered":"2024-09-21T07:27:11.417686+0000","last_unstale":"2024-09-21T07:27:11.420887+0000","last_undegraded":"2024-09-21T07:27:11.420887+0000","last_fullsized":"2024-09-21T07:27:11.420887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:13.784958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156809+0000","last_change":"2024-09-21T07:27:11.420211+0000","last_active":"2024-09-21T07:27:13.156809+0000","last_peered":"2024-09-21T07:27:13.156809+0000","last_clean":"2024-09-21T07:27:13.156809+0000","last_became_active":"2024-09-21T07:27:11.419856+0000","last_became_peered":"2024-09-21T07:27:11.419856+0000","last_unstale":"2024-09-21T07:27:13.156809+0000","last_undegraded":"2024-09-21T07:27:13.156809+0000","last_fullsized":"2024-09-21T07:27:13.156809+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:32.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409176+0000","last_change":"2024-09-21T07:27:11.426553+0000","last_active":"2024-09-21T07:27:12.409176+0000","last_peered":"2024-09-21T07:27:12.409176+0000","last_clean":"2024-09-21T07:27:12.409176+0000","last_became_active":"2024-09-21T07:27:11.426234+0000","last_became_peered":"2024-09-21T07:27:11.426234+0000","last_unstale":"2024-09-21T07:27:12.409176+0000","last_undegraded":"2024-09-21T07:27:12.409176+0000","last_fullsized":"2024-09-21T07:27:12.409176+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:30.080550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409292+0000","last_change":"2024-09-21T07:27:11.419680+0000","last_active":"2024-09-21T07:27:12.409292+0000","last_peered":"2024-09-21T07:27:12.409292+0000","last_clean":"2024-09-21T07:27:12.409292+0000","last_became_active":"2024-09-21T07:27:11.419329+0000","last_became_peered":"2024-09-21T07:27:11.419329+0000","last_unstale":"2024-09-21T07:27:12.409292+0000","last_undegraded":"2024-09-21T07:27:12.409292+0000","last_fullsized":"2024-09-21T07:27:12.409292+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:46.580975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156601+0000","last_change":"2024-09-21T07:27:11.398837+0000","last_active":"2024-09-21T07:27:13.156601+0000","last_peered":"2024-09-21T07:27:13.156601+0000","last_clean":"2024-09-21T07:27:13.156601+0000","last_became_active":"2024-09-21T07:27:11.398312+0000","last_became_peered":"2024-09-21T07:27:11.398312+0000","last_unstale":"2024-09-21T07:27:13.156601+0000","last_undegraded":"2024-09-21T07:27:13.156601+0000","last_fullsized":"2024-09-21T07:27:13.156601+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:44:11.081801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408854+0000","last_change":"2024-09-21T07:27:11.423489+0000","last_active":"2024-09-21T07:27:12.408854+0000","last_peered":"2024-09-21T07:27:12.408854+0000","last_clean":"2024-09-21T07:27:12.408854+0000","last_became_active":"2024-09-21T07:27:11.422541+0000","last_became_peered":"2024-09-21T07:27:11.422541+0000","last_unstale":"2024-09-21T07:27:12.408854+0000","last_undegraded":"2024-09-21T07:27:12.408854+0000","last_fullsized":"2024-09-21T07:27:12.408854+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:57.859007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156323+0000","last_change":"2024-09-21T07:27:11.409499+0000","last_active":"2024-09-21T07:27:13.156323+0000","last_peered":"2024-09-21T07:27:13.156323+0000","last_clean":"2024-09-21T07:27:13.156323+0000","last_became_active":"2024-09-21T07:27:11.407759+0000","last_became_peered":"2024-09-21T07:27:11.407759+0000","last_unstale":"2024-09-21T07:27:13.156323+0000","last_undegraded":"2024-09-21T07:27:13.156323+0000","last_fullsized":"2024-09-21T07:27:13.156323+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:53.609448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408821+0000","last_change":"2024-09-21T07:27:11.421422+0000","last_active":"2024-09-21T07:27:12.408821+0000","last_peered":"2024-09-21T07:27:12.408821+0000","last_clean":"2024-09-21T07:27:12.408821+0000","last_became_active":"2024-09-21T07:27:11.421098+0000","last_became_peered":"2024-09-21T07:27:11.421098+0000","last_unstale":"2024-09-21T07:27:12.408821+0000","last_undegraded":"2024-09-21T07:27:12.408821+0000","last_fullsized":"2024-09-21T07:27:12.408821+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:02:14.367091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.392890+0000","last_change":"2024-09-21T07:27:11.418588+0000","last_active":"2024-09-21T07:27:12.392890+0000","last_peered":"2024-09-21T07:27:12.392890+0000","last_clean":"2024-09-21T07:27:12.392890+0000","last_became_active":"2024-09-21T07:27:11.418249+0000","last_became_peered":"2024-09-21T07:27:11.418249+0000","last_unstale":"2024-09-21T07:27:12.392890+0000","last_undegraded":"2024-09-21T07:27:12.392890+0000","last_fullsized":"2024-09-21T07:27:12.392890+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:43.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394410+0000","last_change":"2024-09-21T07:27:11.418692+0000","last_active":"2024-09-21T07:27:12.394410+0000","last_peered":"2024-09-21T07:27:12.394410+0000","last_clean":"2024-09-21T07:27:12.394410+0000","last_became_active":"2024-09-21T07:27:11.418373+0000","last_became_peered":"2024-09-21T07:27:11.418373+0000","last_unstale":"2024-09-21T07:27:12.394410+0000","last_undegraded":"2024-09-21T07:27:12.394410+0000","last_fullsized":"2024-09-21T07:27:12.394410+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:47.065755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393593+0000","last_change":"2024-09-21T07:27:11.412224+0000","last_active":"2024-09-21T07:27:12.393593+0000","last_peered":"2024-09-21T07:27:12.393593+0000","last_clean":"2024-09-21T07:27:12.393593+0000","last_became_active":"2024-09-21T07:27:11.411905+0000","last_became_peered":"2024-09-21T07:27:11.411905+0000","last_unstale":"2024-09-21T07:27:12.393593+0000","last_undegraded":"2024-09-21T07:27:12.393593+0000","last_fullsized":"2024-09-21T07:27:12.393593+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:19:59.686478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406679+0000","last_change":"2024-09-21T07:27:11.399147+0000","last_active":"2024-09-21T07:27:12.406679+0000","last_peered":"2024-09-21T07:27:12.406679+0000","last_clean":"2024-09-21T07:27:12.406679+0000","last_became_active":"2024-09-21T07:27:11.398841+0000","last_became_peered":"2024-09-21T07:27:11.398841+0000","last_unstale":"2024-09-21T07:27:12.406679+0000","last_undegraded":"2024-09-21T07:27:12.406679+0000","last_fullsized":"2024-09-21T07:27:12.406679+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:33.399273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393137+0000","last_change":"2024-09-21T07:27:11.411784+0000","last_active":"2024-09-21T07:27:12.393137+0000","last_peered":"2024-09-21T07:27:12.393137+0000","last_clean":"2024-09-21T07:27:12.393137+0000","last_became_active":"2024-09-21T07:27:11.411445+0000","last_became_peered":"2024-09-21T07:27:11.411445+0000","last_unstale":"2024-09-21T07:27:12.393137+0000","last_undegraded":"2024-09-21T07:27:12.393137+0000","last_fullsized":"2024-09-21T07:27:12.393137+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:02.250674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409031+0000","last_change":"2024-09-21T07:27:11.415333+0000","last_active":"2024-09-21T07:27:12.409031+0000","last_peered":"2024-09-21T07:27:12.409031+0000","last_clean":"2024-09-21T07:27:12.409031+0000","last_became_active":"2024-09-21T07:27:11.415040+0000","last_became_peered":"2024-09-21T07:27:11.415040+0000","last_unstale":"2024-09-21T07:27:12.409031+0000","last_undegraded":"2024-09-21T07:27:12.409031+0000","last_fullsized":"2024-09-21T07:27:12.409031+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:39.169804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156738+0000","last_change":"2024-09-21T07:27:11.400287+0000","last_active":"2024-09-21T07:27:13.156738+0000","last_peered":"2024-09-21T07:27:13.156738+0000","last_clean":"2024-09-21T07:27:13.156738+0000","last_became_active":"2024-09-21T07:27:11.399860+0000","last_became_peered":"2024-09-21T07:27:11.399860+0000","last_unstale":"2024-09-21T07:27:13.156738+0000","last_undegraded":"2024-09-21T07:27:13.156738+0000","last_fullsized":"2024-09-21T07:27:13.156738+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:08.394338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393161+0000","last_change":"2024-09-21T07:27:11.409776+0000","last_active":"2024-09-21T07:27:12.393161+0000","last_peered":"2024-09-21T07:27:12.393161+0000","last_clean":"2024-09-21T07:27:12.393161+0000","last_became_active":"2024-09-21T07:27:11.408115+0000","last_became_peered":"2024-09-21T07:27:11.408115+0000","last_unstale":"2024-09-21T07:27:12.393161+0000","last_undegraded":"2024-09-21T07:27:12.393161+0000","last_fullsized":"2024-09-21T07:27:12.393161+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:08.394338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406364+0000","last_change":"2024-09-21T07:27:11.409054+0000","last_active":"2024-09-21T07:27:12.406364+0000","last_peered":"2024-09-21T07:27:12.406364+0000","last_clean":"2024-09-21T07:27:12.406364+0000","last_became_active":"2024-09-21T07:27:11.408746+0000","last_became_peered":"2024-09-21T07:27:11.408746+0000","last_unstale":"2024-09-21T07:27:12.406364+0000","last_undegraded":"2024-09-21T07:27:12.406364+0000","last_fullsized":"2024-09-21T07:27:12.406364+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:02.666412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394078+0000","last_change":"2024-09-21T07:27:11.402272+0000","last_active":"2024-09-21T07:27:12.394078+0000","last_peered":"2024-09-21T07:27:12.394078+0000","last_clean":"2024-09-21T07:27:12.394078+0000","last_became_active":"2024-09-21T07:27:11.401931+0000","last_became_peered":"2024-09-21T07:27:11.401931+0000","last_unstale":"2024-09-21T07:27:12.394078+0000","last_undegraded":"2024-09-21T07:27:12.394078+0000","last_fullsized":"2024-09-21T07:27:12.394078+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:43.798333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157377+0000","last_change":"2024-09-21T07:27:11.417199+0000","last_active":"2024-09-21T07:27:13.157377+0000","last_peered":"2024-09-21T07:27:13.157377+0000","last_clean":"2024-09-21T07:27:13.157377+0000","last_became_active":"2024-09-21T07:27:11.416890+0000","last_became_peered":"2024-09-21T07:27:11.416890+0000","last_unstale":"2024-09-21T07:27:13.157377+0000","last_undegraded":"2024-09-21T07:27:13.157377+0000","last_fullsized":"2024-09-21T07:27:13.157377+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:57.840508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394025+0000","last_change":"2024-09-21T07:27:11.404845+0000","last_active":"2024-09-21T07:27:12.394025+0000","last_peered":"2024-09-21T07:27:12.394025+0000","last_clean":"2024-09-21T07:27:12.394025+0000","last_became_active":"2024-09-21T07:27:11.404526+0000","last_became_peered":"2024-09-21T07:27:11.404526+0000","last_unstale":"2024-09-21T07:27:12.394025+0000","last_undegraded":"2024-09-21T07:27:12.394025+0000","last_fullsized":"2024-09-21T07:27:12.394025+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:51.487806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409211+0000","last_change":"2024-09-21T07:27:11.423871+0000","last_active":"2024-09-21T07:27:12.409211+0000","last_peered":"2024-09-21T07:27:12.409211+0000","last_clean":"2024-09-21T07:27:12.409211+0000","last_became_active":"2024-09-21T07:27:11.421863+0000","last_became_peered":"2024-09-21T07:27:11.421863+0000","last_unstale":"2024-09-21T07:27:12.409211+0000","last_undegraded":"2024-09-21T07:27:12.409211+0000","last_fullsized":"2024-09-21T07:27:12.409211+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:05.756366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393237+0000","last_change":"2024-09-21T07:27:11.401854+0000","last_active":"2024-09-21T07:27:12.393237+0000","last_peered":"2024-09-21T07:27:12.393237+0000","last_clean":"2024-09-21T07:27:12.393237+0000","last_became_active":"2024-09-21T07:27:11.401522+0000","last_became_peered":"2024-09-21T07:27:11.401522+0000","last_unstale":"2024-09-21T07:27:12.393237+0000","last_undegraded":"2024-09-21T07:27:12.393237+0000","last_fullsized":"2024-09-21T07:27:12.393237+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:10.633067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393871+0000","last_change":"2024-09-21T07:27:11.407795+0000","last_active":"2024-09-21T07:27:12.393871+0000","last_peered":"2024-09-21T07:27:12.393871+0000","last_clean":"2024-09-21T07:27:12.393871+0000","last_became_active":"2024-09-21T07:27:11.407441+0000","last_became_peered":"2024-09-21T07:27:11.407441+0000","last_unstale":"2024-09-21T07:27:12.393871+0000","last_undegraded":"2024-09-21T07:27:12.393871+0000","last_fullsized":"2024-09-21T07:27:12.393871+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393418+0000","last_change":"2024-09-21T07:27:11.410069+0000","last_active":"2024-09-21T07:27:12.393418+0000","last_peered":"2024-09-21T07:27:12.393418+0000","last_clean":"2024-09-21T07:27:12.393418+0000","last_became_active":"2024-09-21T07:27:11.408126+0000","last_became_peered":"2024-09-21T07:27:11.408126+0000","last_unstale":"2024-09-21T07:27:12.393418+0000","last_undegraded":"2024-09-21T07:27:12.393418+0000","last_fullsized":"2024-09-21T07:27:12.393418+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:14.321600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157499+0000","last_change":"2024-09-21T07:27:11.400123+0000","last_active":"2024-09-21T07:27:13.157499+0000","last_peered":"2024-09-21T07:27:13.157499+0000","last_clean":"2024-09-21T07:27:13.157499+0000","last_became_active":"2024-09-21T07:27:11.399796+0000","last_became_peered":"2024-09-21T07:27:11.399796+0000","last_unstale":"2024-09-21T07:27:13.157499+0000","last_undegraded":"2024-09-21T07:27:13.157499+0000","last_fullsized":"2024-09-21T07:27:13.157499+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394037+0000","last_change":"2024-09-21T07:27:11.418607+0000","last_active":"2024-09-21T07:27:12.394037+0000","last_peered":"2024-09-21T07:27:12.394037+0000","last_clean":"2024-09-21T07:27:12.394037+0000","last_became_active":"2024-09-21T07:27:11.418311+0000","last_became_peered":"2024-09-21T07:27:11.418311+0000","last_unstale":"2024-09-21T07:27:12.394037+0000","last_undegraded":"2024-09-21T07:27:12.394037+0000","last_fullsized":"2024-09-21T07:27:12.394037+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:14.466327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394061+0000","last_change":"2024-09-21T07:27:10.389895+0000","last_active":"2024-09-21T07:27:12.394061+0000","last_peered":"2024-09-21T07:27:12.394061+0000","last_clean":"2024-09-21T07:27:12.394061+0000","last_became_active":"2024-09-21T07:27:10.389570+0000","last_became_peered":"2024-09-21T07:27:10.389570+0000","last_unstale":"2024-09-21T07:27:12.394061+0000","last_undegraded":"2024-09-21T07:27:12.394061+0000","last_fullsized":"2024-09-21T07:27:12.394061+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.319182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157555+0000","last_change":"2024-09-21T07:27:11.419977+0000","last_active":"2024-09-21T07:27:13.157555+0000","last_peered":"2024-09-21T07:27:13.157555+0000","last_clean":"2024-09-21T07:27:13.157555+0000","last_became_active":"2024-09-21T07:27:11.419565+0000","last_became_peered":"2024-09-21T07:27:11.419565+0000","last_unstale":"2024-09-21T07:27:13.157555+0000","last_undegraded":"2024-09-21T07:27:13.157555+0000","last_fullsized":"2024-09-21T07:27:13.157555+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:38:32.377440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156709+0000","last_change":"2024-09-21T07:27:10.391324+0000","last_active":"2024-09-21T07:27:13.156709+0000","last_peered":"2024-09-21T07:27:13.156709+0000","last_clean":"2024-09-21T07:27:13.156709+0000","last_became_active":"2024-09-21T07:27:10.390663+0000","last_became_peered":"2024-09-21T07:27:10.390663+0000","last_unstale":"2024-09-21T07:27:13.156709+0000","last_undegraded":"2024-09-21T07:27:13.156709+0000","last_fullsized":"2024-09-21T07:27:13.156709+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:48.062789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156631+0000","last_change":"2024-09-21T07:27:11.406879+0000","last_active":"2024-09-21T07:27:13.156631+0000","last_peered":"2024-09-21T07:27:13.156631+0000","last_clean":"2024-09-21T07:27:13.156631+0000","last_became_active":"2024-09-21T07:27:11.406542+0000","last_became_peered":"2024-09-21T07:27:11.406542+0000","last_unstale":"2024-09-21T07:27:13.156631+0000","last_undegraded":"2024-09-21T07:27:13.156631+0000","last_fullsized":"2024-09-21T07:27:13.156631+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157695+0000","last_change":"2024-09-21T07:27:10.396250+0000","last_active":"2024-09-21T07:27:13.157695+0000","last_peered":"2024-09-21T07:27:13.157695+0000","last_clean":"2024-09-21T07:27:13.157695+0000","last_became_active":"2024-09-21T07:27:10.395234+0000","last_became_peered":"2024-09-21T07:27:10.395234+0000","last_unstale":"2024-09-21T07:27:13.157695+0000","last_undegraded":"2024-09-21T07:27:13.157695+0000","last_fullsized":"2024-09-21T07:27:13.157695+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:49.882577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418595+0000","last_change":"2024-09-21T07:27:11.404215+0000","last_active":"2024-09-21T07:27:11.418595+0000","last_peered":"2024-09-21T07:27:11.418595+0000","last_clean":"2024-09-21T07:27:11.418595+0000","last_became_active":"2024-09-21T07:27:11.403888+0000","last_became_peered":"2024-09-21T07:27:11.403888+0000","last_unstale":"2024-09-21T07:27:11.418595+0000","last_undegraded":"2024-09-21T07:27:11.418595+0000","last_fullsized":"2024-09-21T07:27:11.418595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:43.851513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156601+0000","last_change":"2024-09-21T07:27:10.392527+0000","last_active":"2024-09-21T07:27:13.156601+0000","last_peered":"2024-09-21T07:27:13.156601+0000","last_clean":"2024-09-21T07:27:13.156601+0000","last_became_active":"2024-09-21T07:27:10.392193+0000","last_became_peered":"2024-09-21T07:27:10.392193+0000","last_unstale":"2024-09-21T07:27:13.156601+0000","last_undegraded":"2024-09-21T07:27:13.156601+0000","last_fullsized":"2024-09-21T07:27:13.156601+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:39.389464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156508+0000","last_change":"2024-09-21T07:27:11.397331+0000","last_active":"2024-09-21T07:27:13.156508+0000","last_peered":"2024-09-21T07:27:13.156508+0000","last_clean":"2024-09-21T07:27:13.156508+0000","last_became_active":"2024-09-21T07:27:11.396883+0000","last_became_peered":"2024-09-21T07:27:11.396883+0000","last_unstale":"2024-09-21T07:27:13.156508+0000","last_undegraded":"2024-09-21T07:27:13.156508+0000","last_fullsized":"2024-09-21T07:27:13.156508+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409018+0000","last_change":"2024-09-21T07:27:10.395602+0000","last_active":"2024-09-21T07:27:12.409018+0000","last_peered":"2024-09-21T07:27:12.409018+0000","last_clean":"2024-09-21T07:27:12.409018+0000","last_became_active":"2024-09-21T07:27:10.395288+0000","last_became_peered":"2024-09-21T07:27:10.395288+0000","last_unstale":"2024-09-21T07:27:12.409018+0000","last_undegraded":"2024-09-21T07:27:12.409018+0000","last_fullsized":"2024-09-21T07:27:12.409018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:56.243273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409098+0000","last_change":"2024-09-21T07:27:11.416365+0000","last_active":"2024-09-21T07:27:12.409098+0000","last_peered":"2024-09-21T07:27:12.409098+0000","last_clean":"2024-09-21T07:27:12.409098+0000","last_became_active":"2024-09-21T07:27:11.415537+0000","last_became_peered":"2024-09-21T07:27:11.415537+0000","last_unstale":"2024-09-21T07:27:12.409098+0000","last_undegraded":"2024-09-21T07:27:12.409098+0000","last_fullsized":"2024-09-21T07:27:12.409098+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:25.786724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393399+0000","last_change":"2024-09-21T07:27:10.396727+0000","last_active":"2024-09-21T07:27:12.393399+0000","last_peered":"2024-09-21T07:27:12.393399+0000","last_clean":"2024-09-21T07:27:12.393399+0000","last_became_active":"2024-09-21T07:27:10.396337+0000","last_became_peered":"2024-09-21T07:27:10.396337+0000","last_unstale":"2024-09-21T07:27:12.393399+0000","last_undegraded":"2024-09-21T07:27:12.393399+0000","last_fullsized":"2024-09-21T07:27:12.393399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:42.372953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156627+0000","last_change":"2024-09-21T07:27:11.419680+0000","last_active":"2024-09-21T07:27:13.156627+0000","last_peered":"2024-09-21T07:27:13.156627+0000","last_clean":"2024-09-21T07:27:13.156627+0000","last_became_active":"2024-09-21T07:27:11.419364+0000","last_became_peered":"2024-09-21T07:27:11.419364+0000","last_unstale":"2024-09-21T07:27:13.156627+0000","last_undegraded":"2024-09-21T07:27:13.156627+0000","last_fullsized":"2024-09-21T07:27:13.156627+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:43.898953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157075+0000","last_change":"2024-09-21T07:27:10.398502+0000","last_active":"2024-09-21T07:27:13.157075+0000","last_peered":"2024-09-21T07:27:13.157075+0000","last_clean":"2024-09-21T07:27:13.157075+0000","last_became_active":"2024-09-21T07:27:10.398173+0000","last_became_peered":"2024-09-21T07:27:10.398173+0000","last_unstale":"2024-09-21T07:27:13.157075+0000","last_undegraded":"2024-09-21T07:27:13.157075+0000","last_fullsized":"2024-09-21T07:27:13.157075+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:43.323314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418635+0000","last_change":"2024-09-21T07:27:11.403570+0000","last_active":"2024-09-21T07:27:11.418635+0000","last_peered":"2024-09-21T07:27:11.418635+0000","last_clean":"2024-09-21T07:27:11.418635+0000","last_became_active":"2024-09-21T07:27:11.403259+0000","last_became_peered":"2024-09-21T07:27:11.403259+0000","last_unstale":"2024-09-21T07:27:11.418635+0000","last_undegraded":"2024-09-21T07:27:11.418635+0000","last_fullsized":"2024-09-21T07:27:11.418635+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:41.232891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156196+0000","last_change":"2024-09-21T07:27:10.387763+0000","last_active":"2024-09-21T07:27:13.156196+0000","last_peered":"2024-09-21T07:27:13.156196+0000","last_clean":"2024-09-21T07:27:13.156196+0000","last_became_active":"2024-09-21T07:27:10.386289+0000","last_became_peered":"2024-09-21T07:27:10.386289+0000","last_unstale":"2024-09-21T07:27:13.156196+0000","last_undegraded":"2024-09-21T07:27:13.156196+0000","last_fullsized":"2024-09-21T07:27:13.156196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:57.843041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406560+0000","last_change":"2024-09-21T07:27:11.408508+0000","last_active":"2024-09-21T07:27:12.406560+0000","last_peered":"2024-09-21T07:27:12.406560+0000","last_clean":"2024-09-21T07:27:12.406560+0000","last_became_active":"2024-09-21T07:27:11.408199+0000","last_became_peered":"2024-09-21T07:27:11.408199+0000","last_unstale":"2024-09-21T07:27:12.406560+0000","last_undegraded":"2024-09-21T07:27:12.406560+0000","last_fullsized":"2024-09-21T07:27:12.406560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:28.179893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.388759+0000","last_change":"2024-09-21T07:27:10.388613+0000","last_active":"2024-09-21T07:27:11.388759+0000","last_peered":"2024-09-21T07:27:11.388759+0000","last_clean":"2024-09-21T07:27:11.388759+0000","last_became_active":"2024-09-21T07:27:10.388295+0000","last_became_peered":"2024-09-21T07:27:10.388295+0000","last_unstale":"2024-09-21T07:27:11.388759+0000","last_undegraded":"2024-09-21T07:27:11.388759+0000","last_fullsized":"2024-09-21T07:27:11.388759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:52.685938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394547+0000","last_change":"2024-09-21T07:27:11.410517+0000","last_active":"2024-09-21T07:27:12.394547+0000","last_peered":"2024-09-21T07:27:12.394547+0000","last_clean":"2024-09-21T07:27:12.394547+0000","last_became_active":"2024-09-21T07:27:11.410184+0000","last_became_peered":"2024-09-21T07:27:11.410184+0000","last_unstale":"2024-09-21T07:27:12.394547+0000","last_undegraded":"2024-09-21T07:27:12.394547+0000","last_fullsized":"2024-09-21T07:27:12.394547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:24.360375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409433+0000","last_change":"2024-09-21T07:27:10.395921+0000","last_active":"2024-09-21T07:27:12.409433+0000","last_peered":"2024-09-21T07:27:12.409433+0000","last_clean":"2024-09-21T07:27:12.409433+0000","last_became_active":"2024-09-21T07:27:10.394941+0000","last_became_peered":"2024-09-21T07:27:10.394941+0000","last_unstale":"2024-09-21T07:27:12.409433+0000","last_undegraded":"2024-09-21T07:27:12.409433+0000","last_fullsized":"2024-09-21T07:27:12.409433+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:32.180578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157104+0000","last_change":"2024-09-21T07:27:11.419437+0000","last_active":"2024-09-21T07:27:13.157104+0000","last_peered":"2024-09-21T07:27:13.157104+0000","last_clean":"2024-09-21T07:27:13.157104+0000","last_became_active":"2024-09-21T07:27:11.419125+0000","last_became_peered":"2024-09-21T07:27:11.419125+0000","last_unstale":"2024-09-21T07:27:13.157104+0000","last_undegraded":"2024-09-21T07:27:13.157104+0000","last_fullsized":"2024-09-21T07:27:13.157104+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:40.507567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409430+0000","last_change":"2024-09-21T07:27:10.395815+0000","last_active":"2024-09-21T07:27:12.409430+0000","last_peered":"2024-09-21T07:27:12.409430+0000","last_clean":"2024-09-21T07:27:12.409430+0000","last_became_active":"2024-09-21T07:27:10.395395+0000","last_became_peered":"2024-09-21T07:27:10.395395+0000","last_unstale":"2024-09-21T07:27:12.409430+0000","last_undegraded":"2024-09-21T07:27:12.409430+0000","last_fullsized":"2024-09-21T07:27:12.409430+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:29.237028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156673+0000","last_change":"2024-09-21T07:27:11.398272+0000","last_active":"2024-09-21T07:27:13.156673+0000","last_peered":"2024-09-21T07:27:13.156673+0000","last_clean":"2024-09-21T07:27:13.156673+0000","last_became_active":"2024-09-21T07:27:11.397821+0000","last_became_peered":"2024-09-21T07:27:11.397821+0000","last_unstale":"2024-09-21T07:27:13.156673+0000","last_undegraded":"2024-09-21T07:27:13.156673+0000","last_fullsized":"2024-09-21T07:27:13.156673+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:53.111009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.428317+0000","last_change":"2024-09-21T07:27:10.391438+0000","last_active":"2024-09-21T07:27:11.428317+0000","last_peered":"2024-09-21T07:27:11.428317+0000","last_clean":"2024-09-21T07:27:11.428317+0000","last_became_active":"2024-09-21T07:27:10.391138+0000","last_became_peered":"2024-09-21T07:27:10.391138+0000","last_unstale":"2024-09-21T07:27:11.428317+0000","last_undegraded":"2024-09-21T07:27:11.428317+0000","last_fullsized":"2024-09-21T07:27:11.428317+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:41.148308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156775+0000","last_change":"2024-09-21T07:27:11.416980+0000","last_active":"2024-09-21T07:27:13.156775+0000","last_peered":"2024-09-21T07:27:13.156775+0000","last_clean":"2024-09-21T07:27:13.156775+0000","last_became_active":"2024-09-21T07:27:11.416627+0000","last_became_peered":"2024-09-21T07:27:11.416627+0000","last_unstale":"2024-09-21T07:27:13.156775+0000","last_undegraded":"2024-09-21T07:27:13.156775+0000","last_fullsized":"2024-09-21T07:27:13.156775+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:14.412733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420702+0000","last_change":"2024-09-21T07:27:10.392664+0000","last_active":"2024-09-21T07:27:11.420702+0000","last_peered":"2024-09-21T07:27:11.420702+0000","last_clean":"2024-09-21T07:27:11.420702+0000","last_became_active":"2024-09-21T07:27:10.392208+0000","last_became_peered":"2024-09-21T07:27:10.392208+0000","last_unstale":"2024-09-21T07:27:11.420702+0000","last_undegraded":"2024-09-21T07:27:11.420702+0000","last_fullsized":"2024-09-21T07:27:11.420702+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:53.537965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.405059+0000","last_change":"2024-09-21T07:27:11.400024+0000","last_active":"2024-09-21T07:27:11.405059+0000","last_peered":"2024-09-21T07:27:11.405059+0000","last_clean":"2024-09-21T07:27:11.405059+0000","last_became_active":"2024-09-21T07:27:11.399714+0000","last_became_peered":"2024-09-21T07:27:11.399714+0000","last_unstale":"2024-09-21T07:27:11.405059+0000","last_undegraded":"2024-09-21T07:27:11.405059+0000","last_fullsized":"2024-09-21T07:27:11.405059+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:58.985253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157213+0000","last_change":"2024-09-21T07:27:10.397596+0000","last_active":"2024-09-21T07:27:13.157213+0000","last_peered":"2024-09-21T07:27:13.157213+0000","last_clean":"2024-09-21T07:27:13.157213+0000","last_became_active":"2024-09-21T07:27:10.397245+0000","last_became_peered":"2024-09-21T07:27:10.397245+0000","last_unstale":"2024-09-21T07:27:13.157213+0000","last_undegraded":"2024-09-21T07:27:13.157213+0000","last_fullsized":"2024-09-21T07:27:13.157213+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:04:48.062789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393142+0000","last_change":"2024-09-21T07:27:11.409267+0000","last_active":"2024-09-21T07:27:12.393142+0000","last_peered":"2024-09-21T07:27:12.393142+0000","last_clean":"2024-09-21T07:27:12.393142+0000","last_became_active":"2024-09-21T07:27:11.408350+0000","last_became_peered":"2024-09-21T07:27:11.408350+0000","last_unstale":"2024-09-21T07:27:12.393142+0000","last_undegraded":"2024-09-21T07:27:12.393142+0000","last_fullsized":"2024-09-21T07:27:12.393142+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:53.111009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.392828+0000","last_change":"2024-09-21T07:27:10.386578+0000","last_active":"2024-09-21T07:27:12.392828+0000","last_peered":"2024-09-21T07:27:12.392828+0000","last_clean":"2024-09-21T07:27:12.392828+0000","last_became_active":"2024-09-21T07:27:10.386265+0000","last_became_peered":"2024-09-21T07:27:10.386265+0000","last_unstale":"2024-09-21T07:27:12.392828+0000","last_undegraded":"2024-09-21T07:27:12.392828+0000","last_fullsized":"2024-09-21T07:27:12.392828+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:32.393599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.407427+0000","last_change":"2024-09-21T07:27:11.400834+0000","last_active":"2024-09-21T07:27:11.407427+0000","last_peered":"2024-09-21T07:27:11.407427+0000","last_clean":"2024-09-21T07:27:11.407427+0000","last_became_active":"2024-09-21T07:27:11.400546+0000","last_became_peered":"2024-09-21T07:27:11.400546+0000","last_unstale":"2024-09-21T07:27:11.407427+0000","last_undegraded":"2024-09-21T07:27:11.407427+0000","last_fullsized":"2024-09-21T07:27:11.407427+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:30.482260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393238+0000","last_change":"2024-09-21T07:27:10.397744+0000","last_active":"2024-09-21T07:27:12.393238+0000","last_peered":"2024-09-21T07:27:12.393238+0000","last_clean":"2024-09-21T07:27:12.393238+0000","last_became_active":"2024-09-21T07:27:10.397379+0000","last_became_peered":"2024-09-21T07:27:10.397379+0000","last_unstale":"2024-09-21T07:27:12.393238+0000","last_undegraded":"2024-09-21T07:27:12.393238+0000","last_fullsized":"2024-09-21T07:27:12.393238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:57.843041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.420282+0000","last_change":"2024-09-21T07:27:11.407006+0000","last_active":"2024-09-21T07:27:11.420282+0000","last_peered":"2024-09-21T07:27:11.420282+0000","last_clean":"2024-09-21T07:27:11.420282+0000","last_became_active":"2024-09-21T07:27:11.406669+0000","last_became_peered":"2024-09-21T07:27:11.406669+0000","last_unstale":"2024-09-21T07:27:11.420282+0000","last_undegraded":"2024-09-21T07:27:11.420282+0000","last_fullsized":"2024-09-21T07:27:11.420282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:19.675326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409480+0000","last_change":"2024-09-21T07:27:10.397819+0000","last_active":"2024-09-21T07:27:12.409480+0000","last_peered":"2024-09-21T07:27:12.409480+0000","last_clean":"2024-09-21T07:27:12.409480+0000","last_became_active":"2024-09-21T07:27:10.397418+0000","last_became_peered":"2024-09-21T07:27:10.397418+0000","last_unstale":"2024-09-21T07:27:12.409480+0000","last_undegraded":"2024-09-21T07:27:12.409480+0000","last_fullsized":"2024-09-21T07:27:12.409480+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:30.816349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409362+0000","last_change":"2024-09-21T07:27:11.417488+0000","last_active":"2024-09-21T07:27:12.409362+0000","last_peered":"2024-09-21T07:27:12.409362+0000","last_clean":"2024-09-21T07:27:12.409362+0000","last_became_active":"2024-09-21T07:27:11.416782+0000","last_became_peered":"2024-09-21T07:27:11.416782+0000","last_unstale":"2024-09-21T07:27:12.409362+0000","last_undegraded":"2024-09-21T07:27:12.409362+0000","last_fullsized":"2024-09-21T07:27:12.409362+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:41.958678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156768+0000","last_change":"2024-09-21T07:27:10.395859+0000","last_active":"2024-09-21T07:27:13.156768+0000","last_peered":"2024-09-21T07:27:13.156768+0000","last_clean":"2024-09-21T07:27:13.156768+0000","last_became_active":"2024-09-21T07:27:10.395549+0000","last_became_peered":"2024-09-21T07:27:10.395549+0000","last_unstale":"2024-09-21T07:27:13.156768+0000","last_undegraded":"2024-09-21T07:27:13.156768+0000","last_fullsized":"2024-09-21T07:27:13.156768+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:56.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156600+0000","last_change":"2024-09-21T07:27:11.399404+0000","last_active":"2024-09-21T07:27:13.156600+0000","last_peered":"2024-09-21T07:27:13.156600+0000","last_clean":"2024-09-21T07:27:13.156600+0000","last_became_active":"2024-09-21T07:27:11.398941+0000","last_became_peered":"2024-09-21T07:27:11.398941+0000","last_unstale":"2024-09-21T07:27:13.156600+0000","last_undegraded":"2024-09-21T07:27:13.156600+0000","last_fullsized":"2024-09-21T07:27:13.156600+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:07:14.516851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406597+0000","last_change":"2024-09-21T07:27:10.396054+0000","last_active":"2024-09-21T07:27:12.406597+0000","last_peered":"2024-09-21T07:27:12.406597+0000","last_clean":"2024-09-21T07:27:12.406597+0000","last_became_active":"2024-09-21T07:27:10.395716+0000","last_became_peered":"2024-09-21T07:27:10.395716+0000","last_unstale":"2024-09-21T07:27:12.406597+0000","last_undegraded":"2024-09-21T07:27:12.406597+0000","last_fullsized":"2024-09-21T07:27:12.406597+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:56:25.025889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406737+0000","last_change":"2024-09-21T07:27:11.397987+0000","last_active":"2024-09-21T07:27:12.406737+0000","last_peered":"2024-09-21T07:27:12.406737+0000","last_clean":"2024-09-21T07:27:12.406737+0000","last_became_active":"2024-09-21T07:27:11.397691+0000","last_became_peered":"2024-09-21T07:27:11.397691+0000","last_unstale":"2024-09-21T07:27:12.406737+0000","last_undegraded":"2024-09-21T07:27:12.406737+0000","last_fullsized":"2024-09-21T07:27:12.406737+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:43.187380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157356+0000","last_change":"2024-09-21T07:27:10.398432+0000","last_active":"2024-09-21T07:27:13.157356+0000","last_peered":"2024-09-21T07:27:13.157356+0000","last_clean":"2024-09-21T07:27:13.157356+0000","last_became_active":"2024-09-21T07:27:10.397959+0000","last_became_peered":"2024-09-21T07:27:10.397959+0000","last_unstale":"2024-09-21T07:27:13.157356+0000","last_undegraded":"2024-09-21T07:27:13.157356+0000","last_fullsized":"2024-09-21T07:27:13.157356+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:45.381411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393115+0000","last_change":"2024-09-21T07:27:11.414577+0000","last_active":"2024-09-21T07:27:12.393115+0000","last_peered":"2024-09-21T07:27:12.393115+0000","last_clean":"2024-09-21T07:27:12.393115+0000","last_became_active":"2024-09-21T07:27:11.414248+0000","last_became_peered":"2024-09-21T07:27:11.414248+0000","last_unstale":"2024-09-21T07:27:12.393115+0000","last_undegraded":"2024-09-21T07:27:12.393115+0000","last_fullsized":"2024-09-21T07:27:12.393115+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:00.795137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406452+0000","last_change":"2024-09-21T07:27:10.395227+0000","last_active":"2024-09-21T07:27:12.406452+0000","last_peered":"2024-09-21T07:27:12.406452+0000","last_clean":"2024-09-21T07:27:12.406452+0000","last_became_active":"2024-09-21T07:27:10.394862+0000","last_became_peered":"2024-09-21T07:27:10.394862+0000","last_unstale":"2024-09-21T07:27:12.406452+0000","last_undegraded":"2024-09-21T07:27:12.406452+0000","last_fullsized":"2024-09-21T07:27:12.406452+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:39.418324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156545+0000","last_change":"2024-09-21T07:27:11.402672+0000","last_active":"2024-09-21T07:27:13.156545+0000","last_peered":"2024-09-21T07:27:13.156545+0000","last_clean":"2024-09-21T07:27:13.156545+0000","last_became_active":"2024-09-21T07:27:11.402364+0000","last_became_peered":"2024-09-21T07:27:11.402364+0000","last_unstale":"2024-09-21T07:27:13.156545+0000","last_undegraded":"2024-09-21T07:27:13.156545+0000","last_fullsized":"2024-09-21T07:27:13.156545+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:53.609448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406374+0000","last_change":"2024-09-21T07:27:10.396828+0000","last_active":"2024-09-21T07:27:12.406374+0000","last_peered":"2024-09-21T07:27:12.406374+0000","last_clean":"2024-09-21T07:27:12.406374+0000","last_became_active":"2024-09-21T07:27:10.396238+0000","last_became_peered":"2024-09-21T07:27:10.396238+0000","last_unstale":"2024-09-21T07:27:12.406374+0000","last_undegraded":"2024-09-21T07:27:12.406374+0000","last_fullsized":"2024-09-21T07:27:12.406374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:28.076759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.408952+0000","last_change":"2024-09-21T07:27:11.416310+0000","last_active":"2024-09-21T07:27:12.408952+0000","last_peered":"2024-09-21T07:27:12.408952+0000","last_clean":"2024-09-21T07:27:12.408952+0000","last_became_active":"2024-09-21T07:27:11.415973+0000","last_became_peered":"2024-09-21T07:27:11.415973+0000","last_unstale":"2024-09-21T07:27:12.408952+0000","last_undegraded":"2024-09-21T07:27:12.408952+0000","last_fullsized":"2024-09-21T07:27:12.408952+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:57.999550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393539+0000","last_change":"2024-09-21T07:27:10.392540+0000","last_active":"2024-09-21T07:27:12.393539+0000","last_peered":"2024-09-21T07:27:12.393539+0000","last_clean":"2024-09-21T07:27:12.393539+0000","last_became_active":"2024-09-21T07:27:10.392144+0000","last_became_peered":"2024-09-21T07:27:10.392144+0000","last_unstale":"2024-09-21T07:27:12.393539+0000","last_undegraded":"2024-09-21T07:27:12.393539+0000","last_fullsized":"2024-09-21T07:27:12.393539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:39.389464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.418664+0000","last_change":"2024-09-21T07:27:11.404678+0000","last_active":"2024-09-21T07:27:11.418664+0000","last_peered":"2024-09-21T07:27:11.418664+0000","last_clean":"2024-09-21T07:27:11.418664+0000","last_became_active":"2024-09-21T07:27:11.404385+0000","last_became_peered":"2024-09-21T07:27:11.404385+0000","last_unstale":"2024-09-21T07:27:11.418664+0000","last_undegraded":"2024-09-21T07:27:11.418664+0000","last_fullsized":"2024-09-21T07:27:11.418664+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:37:24.952994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.406289+0000","last_change":"2024-09-21T07:27:10.394546+0000","last_active":"2024-09-21T07:27:12.406289+0000","last_peered":"2024-09-21T07:27:12.406289+0000","last_clean":"2024-09-21T07:27:12.406289+0000","last_became_active":"2024-09-21T07:27:10.394098+0000","last_became_peered":"2024-09-21T07:27:10.394098+0000","last_unstale":"2024-09-21T07:27:12.406289+0000","last_undegraded":"2024-09-21T07:27:12.406289+0000","last_fullsized":"2024-09-21T07:27:12.406289+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:46.231370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157074+0000","last_change":"2024-09-21T07:27:11.419603+0000","last_active":"2024-09-21T07:27:13.157074+0000","last_peered":"2024-09-21T07:27:13.157074+0000","last_clean":"2024-09-21T07:27:13.157074+0000","last_became_active":"2024-09-21T07:27:11.419312+0000","last_became_peered":"2024-09-21T07:27:11.419312+0000","last_unstale":"2024-09-21T07:27:13.157074+0000","last_undegraded":"2024-09-21T07:27:13.157074+0000","last_fullsized":"2024-09-21T07:27:13.157074+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.508952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409547+0000","last_change":"2024-09-21T07:27:10.393397+0000","last_active":"2024-09-21T07:27:12.409547+0000","last_peered":"2024-09-21T07:27:12.409547+0000","last_clean":"2024-09-21T07:27:12.409547+0000","last_became_active":"2024-09-21T07:27:10.393068+0000","last_became_peered":"2024-09-21T07:27:10.393068+0000","last_unstale":"2024-09-21T07:27:12.409547+0000","last_undegraded":"2024-09-21T07:27:12.409547+0000","last_fullsized":"2024-09-21T07:27:12.409547+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:19.651643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394534+0000","last_change":"2024-09-21T07:27:11.411931+0000","last_active":"2024-09-21T07:27:12.394534+0000","last_peered":"2024-09-21T07:27:12.394534+0000","last_clean":"2024-09-21T07:27:12.394534+0000","last_became_active":"2024-09-21T07:27:11.411616+0000","last_became_peered":"2024-09-21T07:27:11.411616+0000","last_unstale":"2024-09-21T07:27:12.394534+0000","last_undegraded":"2024-09-21T07:27:12.394534+0000","last_fullsized":"2024-09-21T07:27:12.394534+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:03.851132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.407918+0000","last_change":"2024-09-21T07:27:10.398924+0000","last_active":"2024-09-21T07:27:12.407918+0000","last_peered":"2024-09-21T07:27:12.407918+0000","last_clean":"2024-09-21T07:27:12.407918+0000","last_became_active":"2024-09-21T07:27:10.398676+0000","last_became_peered":"2024-09-21T07:27:10.398676+0000","last_unstale":"2024-09-21T07:27:12.407918+0000","last_undegraded":"2024-09-21T07:27:12.407918+0000","last_fullsized":"2024-09-21T07:27:12.407918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:29:18.062836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.407902+0000","last_change":"2024-09-21T07:27:11.402449+0000","last_active":"2024-09-21T07:27:11.407902+0000","last_peered":"2024-09-21T07:27:11.407902+0000","last_clean":"2024-09-21T07:27:11.407902+0000","last_became_active":"2024-09-21T07:27:11.402178+0000","last_became_peered":"2024-09-21T07:27:11.402178+0000","last_unstale":"2024-09-21T07:27:11.407902+0000","last_undegraded":"2024-09-21T07:27:11.407902+0000","last_fullsized":"2024-09-21T07:27:11.407902+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:15:31.567351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156724+0000","last_change":"2024-09-21T07:27:10.390528+0000","last_active":"2024-09-21T07:27:13.156724+0000","last_peered":"2024-09-21T07:27:13.156724+0000","last_clean":"2024-09-21T07:27:13.156724+0000","last_became_active":"2024-09-21T07:27:10.390230+0000","last_became_peered":"2024-09-21T07:27:10.390230+0000","last_unstale":"2024-09-21T07:27:13.156724+0000","last_undegraded":"2024-09-21T07:27:13.156724+0000","last_fullsized":"2024-09-21T07:27:13.156724+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:55:12.319182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156834+0000","last_change":"2024-09-21T07:27:11.406593+0000","last_active":"2024-09-21T07:27:13.156834+0000","last_peered":"2024-09-21T07:27:13.156834+0000","last_clean":"2024-09-21T07:27:13.156834+0000","last_became_active":"2024-09-21T07:27:11.406229+0000","last_became_peered":"2024-09-21T07:27:11.406229+0000","last_unstale":"2024-09-21T07:27:13.156834+0000","last_undegraded":"2024-09-21T07:27:13.156834+0000","last_fullsized":"2024-09-21T07:27:13.156834+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:33.803577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157045+0000","last_change":"2024-09-21T07:27:10.390195+0000","last_active":"2024-09-21T07:27:13.157045+0000","last_peered":"2024-09-21T07:27:13.157045+0000","last_clean":"2024-09-21T07:27:13.157045+0000","last_became_active":"2024-09-21T07:27:10.389881+0000","last_became_peered":"2024-09-21T07:27:10.389881+0000","last_unstale":"2024-09-21T07:27:13.157045+0000","last_undegraded":"2024-09-21T07:27:13.157045+0000","last_fullsized":"2024-09-21T07:27:13.157045+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:40:45.381411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.157341+0000","last_change":"2024-09-21T07:27:11.405867+0000","last_active":"2024-09-21T07:27:13.157341+0000","last_peered":"2024-09-21T07:27:13.157341+0000","last_clean":"2024-09-21T07:27:13.157341+0000","last_became_active":"2024-09-21T07:27:11.405535+0000","last_became_peered":"2024-09-21T07:27:11.405535+0000","last_unstale":"2024-09-21T07:27:13.157341+0000","last_undegraded":"2024-09-21T07:27:13.157341+0000","last_fullsized":"2024-09-21T07:27:13.157341+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:21.318618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393739+0000","last_change":"2024-09-21T07:27:10.391250+0000","last_active":"2024-09-21T07:27:12.393739+0000","last_peered":"2024-09-21T07:27:12.393739+0000","last_clean":"2024-09-21T07:27:12.393739+0000","last_became_active":"2024-09-21T07:27:10.390890+0000","last_became_peered":"2024-09-21T07:27:10.390890+0000","last_unstale":"2024-09-21T07:27:12.393739+0000","last_undegraded":"2024-09-21T07:27:12.393739+0000","last_fullsized":"2024-09-21T07:27:12.393739+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:42.181705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156373+0000","last_change":"2024-09-21T07:27:11.406895+0000","last_active":"2024-09-21T07:27:13.156373+0000","last_peered":"2024-09-21T07:27:13.156373+0000","last_clean":"2024-09-21T07:27:13.156373+0000","last_became_active":"2024-09-21T07:27:11.406525+0000","last_became_peered":"2024-09-21T07:27:11.406525+0000","last_unstale":"2024-09-21T07:27:13.156373+0000","last_undegraded":"2024-09-21T07:27:13.156373+0000","last_fullsized":"2024-09-21T07:27:13.156373+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:21.318618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:13.156179+0000","last_change":"2024-09-21T07:27:10.387436+0000","last_active":"2024-09-21T07:27:13.156179+0000","last_peered":"2024-09-21T07:27:13.156179+0000","last_clean":"2024-09-21T07:27:13.156179+0000","last_became_active":"2024-09-21T07:27:10.386526+0000","last_became_peered":"2024-09-21T07:27:10.386526+0000","last_unstale":"2024-09-21T07:27:13.156179+0000","last_undegraded":"2024-09-21T07:27:13.156179+0000","last_fullsized":"2024-09-21T07:27:13.156179+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:42.372953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394310+0000","last_change":"2024-09-21T07:27:11.411274+0000","last_active":"2024-09-21T07:27:12.394310+0000","last_peered":"2024-09-21T07:27:12.394310+0000","last_clean":"2024-09-21T07:27:12.394310+0000","last_became_active":"2024-09-21T07:27:11.410821+0000","last_became_peered":"2024-09-21T07:27:11.410821+0000","last_unstale":"2024-09-21T07:27:12.394310+0000","last_undegraded":"2024-09-21T07:27:12.394310+0000","last_fullsized":"2024-09-21T07:27:12.394310+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:39.773957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-21T07:27:11.387814+0000","last_change":"2024-09-21T07:27:10.387628+0000","last_active":"2024-09-21T07:27:11.387814+0000","last_peered":"2024-09-21T07:27:11.387814+0000","last_clean":"2024-09-21T07:27:11.387814+0000","last_became_active":"2024-09-21T07:27:10.387344+0000","last_became_peered":"2024-09-21T07:27:10.387344+0000","last_unstale":"2024-09-21T07:27:11.387814+0000","last_undegraded":"2024-09-21T07:27:11.387814+0000","last_fullsized":"2024-09-21T07:27:11.387814+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:30.294147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.409446+0000","last_change":"2024-09-21T07:27:11.422387+0000","last_active":"2024-09-21T07:27:12.409446+0000","last_peered":"2024-09-21T07:27:12.409446+0000","last_clean":"2024-09-21T07:27:12.409446+0000","last_became_active":"2024-09-21T07:27:11.422059+0000","last_became_peered":"2024-09-21T07:27:11.422059+0000","last_unstale":"2024-09-21T07:27:12.409446+0000","last_undegraded":"2024-09-21T07:27:12.409446+0000","last_fullsized":"2024-09-21T07:27:12.409446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:10.370208+0000","last_clean_scrub_stamp":"2024-09-21T07:27:10.370208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:53:16.559852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.394273+0000","last_change":"2024-09-21T07:27:10.392863+0000","last_active":"2024-09-21T07:27:12.394273+0000","last_peered":"2024-09-21T07:27:12.394273+0000","last_clean":"2024-09-21T07:27:12.394273+0000","last_became_active":"2024-09-21T07:27:10.392554+0000","last_became_peered":"2024-09-21T07:27:10.392554+0000","last_unstale":"2024-09-21T07:27:12.394273+0000","last_undegraded":"2024-09-21T07:27:12.394273+0000","last_fullsized":"2024-09-21T07:27:12.394273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:27:09.364533+0000","last_clean_scrub_stamp":"2024-09-21T07:27:09.364533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:56.761112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":148,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-21T07:27:12.393182+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:12.393182+0000","last_peered":"2024-09-21T07:27:12.393182+0000","last_clean":"2024-09-21T07:27:12.393182+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:12.393182+0000","last_undegraded":"2024-09-21T07:27:12.393182+0000","last_fullsized":"2024-09-21T07:27:12.393182+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":795717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":204249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":203779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":475136,"data_stored":204271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":203737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":203757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":794069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":196975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:15.772 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:27:15.772 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:27:15.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 -- 172.21.15.105:0/949747640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89800053a0 msgr2=0x7f898000ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 --2- 172.21.15.105:0/949747640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89800053a0 0x7f898000ba50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f899000baf0 tx=0x7f8980005830 comp rx=0 tx=0).stop 2024-09-21T07:27:15.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 -- 172.21.15.105:0/949747640 shutdown_connections 2024-09-21T07:27:15.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 --2- 172.21.15.105:0/949747640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89800053a0 0x7f898000ba50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 -- 172.21.15.105:0/949747640 >> 172.21.15.105:0/949747640 conn(0x7f899809d830 msgr2=0x7f89980d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:15.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 -- 172.21.15.105:0/949747640 shutdown_connections 2024-09-21T07:27:15.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.927+0000 7f899d4dc640 1 -- 172.21.15.105:0/949747640 wait complete. 2024-09-21T07:27:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.929+0000 7f899d4dc640 1 Processor -- start 2024-09-21T07:27:15.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.929+0000 7f899d4dc640 1 -- start start 2024-09-21T07:27:15.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.930+0000 7f899d4dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89800053a0 0x7f89980c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.930+0000 7f899d4dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.930+0000 7f899d4dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89981f4ee0 0x7f89980ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f899d4dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f89980b63b0 con 0x7f89981f4ad0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f899d4dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89980b6230 con 0x7f89800053a0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f899d4dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f89980b6530 con 0x7f89981f4ee0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58356/0 (socket says 172.21.15.105:58356) 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 -- 172.21.15.105:0/2083315140 learned_addr learned my addr 172.21.15.105:0/2083315140 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89977fe640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89981f4ee0 0x7f89980ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f8996ffd640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89800053a0 0x7f89980c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 -- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89981f4ee0 msgr2=0x7f89980ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89981f4ee0 0x7f89980ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 -- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89800053a0 msgr2=0x7f89980c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:15.934 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89800053a0 0x7f89980c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 -- 172.21.15.105:0/2083315140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89980d1800 con 0x7f89981f4ad0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f89967fc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f898c004dc0 tx=0x7f898c001620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c0105d0 con 0x7f89981f4ad0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f898c016070 con 0x7f89981f4ad0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89980d1ae0 con 0x7f89981f4ad0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c010ae0 con 0x7f89981f4ad0 2024-09-21T07:27:15.935 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.931+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89980d20a0 con 0x7f89981f4ad0 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.932+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899806de60 con 0x7f89981f4ad0 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.933+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f898c010c80 con 0x7f89981f4ad0 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.934+0000 7f897f7fe640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 0x7f898407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.934+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f898c09b4c0 con 0x7f89981f4ad0 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.934+0000 7f8996ffd640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 0x7f898407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:15.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.935+0000 7f8996ffd640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 0x7f898407b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8988004c40 tx=0x7f8988006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:15.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:15.938+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f898c0631b0 con 0x7f89981f4ad0 2024-09-21T07:27:16.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.187+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f89980c7de0 con 0x7f89981f4ad0 2024-09-21T07:27:16.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.189+0000 7f897f7fe640 1 -- 172.21.15.105:0/2083315140 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f898c067be0 con 0x7f89981f4ad0 2024-09-21T07:27:16.191 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-09-21T07:27:16.192 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQDbdO5mp/ImCxAASyikrNfrfPlhCoeUsnBKSA== 2024-09-21T07:27:16.192 INFO:teuthology.orchestra.run.smithi105.stdout: caps mds = "allow" 2024-09-21T07:27:16.192 INFO:teuthology.orchestra.run.smithi105.stdout: caps mon = "allow r" 2024-09-21T07:27:16.192 INFO:teuthology.orchestra.run.smithi105.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:27:16.192 INFO:teuthology.orchestra.run.smithi105.stderr:updated caps for client.0 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 msgr2=0x7f898407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 0x7f898407b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8988004c40 tx=0x7f8988006040 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 msgr2=0x7f89980ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f898c004dc0 tx=0x7f898c001620 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 shutdown_connections 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8984078b90 0x7f898407b0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f89981f4ee0 0x7f89980ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89981f4ad0 0x7f89980ca450 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 --2- 172.21.15.105:0/2083315140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89800053a0 0x7f89980c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 >> 172.21.15.105:0/2083315140 conn(0x7f899809d830 msgr2=0x7f8998161ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:16.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 shutdown_connections 2024-09-21T07:27:16.195 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.192+0000 7f899d4dc640 1 -- 172.21.15.105:0/2083315140 wait complete. 2024-09-21T07:27:16.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 -- 172.21.15.105:0/1614443996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3800004d40 msgr2=0x7f38000051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 --2- 172.21.15.105:0/1614443996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3800004d40 0x7f38000051f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3810001630 tx=0x7f3800005830 comp rx=0 tx=0).stop 2024-09-21T07:27:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381d107640 1 -- 172.21.15.105:0/1614443996 reap_dead start 2024-09-21T07:27:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 -- 172.21.15.105:0/1614443996 shutdown_connections 2024-09-21T07:27:16.318 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 -- 172.21.15.105:0/1614443996 >> 172.21.15.105:0/1614443996 conn(0x7f38180cee40 msgr2=0x7f38180cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 -- 172.21.15.105:0/1614443996 shutdown_connections 2024-09-21T07:27:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.316+0000 7f381eb6a640 1 -- 172.21.15.105:0/1614443996 wait complete. 2024-09-21T07:27:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.318+0000 7f381eb6a640 1 Processor -- start 2024-09-21T07:27:16.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.318+0000 7f381eb6a640 1 -- start start 2024-09-21T07:27:16.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 0x7f38181f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:16.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:16.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 0x7f38180caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:16.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f38180d0170 con 0x7f38180b1920 2024-09-21T07:27:16.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f381809bc80 con 0x7f38180b68b0 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381eb6a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f38180d02f0 con 0x7f3800004d40 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f380ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f380ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58366/0 (socket says 172.21.15.105:58366) 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f380ffff640 1 -- 172.21.15.105:0/3165776765 learned_addr learned my addr 172.21.15.105:0/3165776765 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.319+0000 7f381c906640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 0x7f38181f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f381d107640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 0x7f38180caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 -- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 msgr2=0x7f38181f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 0x7f38181f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 -- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 msgr2=0x7f38180caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 0x7f38180caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 -- 172.21.15.105:0/3165776765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180b6cc0 con 0x7f38180b1920 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f381c906640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 0x7f38181f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f381d107640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 0x7f38180caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:16.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380ffff640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38080066e0 tx=0x7f3808003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800f780 con 0x7f38180b1920 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808016020 con 0x7f38180b1920 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38180ca560 con 0x7f38180b1920 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380800fc00 con 0x7f38180b1920 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.320+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38180af430 con 0x7f38180b1920 2024-09-21T07:27:16.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.321+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38180c55d0 con 0x7f38180b1920 2024-09-21T07:27:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.322+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3808011430 con 0x7f38180b1920 2024-09-21T07:27:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.322+0000 7f380d7fa640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 0x7f380407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.322+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f380809b760 con 0x7f38180b1920 2024-09-21T07:27:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.323+0000 7f381c906640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 0x7f380407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:16.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.323+0000 7f381c906640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 0x7f380407b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3814005fd0 tx=0x7f38140094a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:16.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.327+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3808020030 con 0x7f38180b1920 2024-09-21T07:27:16.545 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.543+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38180ce710 con 0x7f38180b1920 2024-09-21T07:27:16.546 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.544+0000 7f380d7fa640 1 -- 172.21.15.105:0/3165776765 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7f38180b6cc0 con 0x7f38180b1920 2024-09-21T07:27:16.546 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:16.546 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:16.547 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.546+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 msgr2=0x7f380407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.546+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 0x7f380407b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3814005fd0 tx=0x7f38140094a0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.546+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 msgr2=0x7f38180b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.546+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38080066e0 tx=0x7f3808003a80 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 shutdown_connections 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f3804078b90 0x7f380407b0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38180b68b0 0x7f38180caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f38180b1920 0x7f38180b6370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 --2- 172.21.15.105:0/3165776765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3800004d40 0x7f38181f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:16.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 >> 172.21.15.105:0/3165776765 conn(0x7f38180cee40 msgr2=0x7f38181526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 shutdown_connections 2024-09-21T07:27:16.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:16.547+0000 7f381eb6a640 1 -- 172.21.15.105:0/3165776765 wait complete. 2024-09-21T07:27:16.962 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:27:10.743141+0000', 'modified': '2024-09-21T07:27:13.773777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5073}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5073': {'gid': 5073, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.105:6833/1752443650', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 1752443650}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 1752443650}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5073, 'qdb_cluster': [5073]} 2024-09-21T07:27:16.963 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:27:16.963 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.076+0000 7f7c2a35e640 1 -- 172.21.15.105:0/1204159625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c240b6440 msgr2=0x7f7c240cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.076+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/1204159625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c240b6440 0x7f7c240cf050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c2408e330 tx=0x7f7c1802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 -- 172.21.15.105:0/1204159625 shutdown_connections 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/1204159625 >> v2:172.21.15.112:3300/0 conn(0x7f7c240b6850 0x7f7c240cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/1204159625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c240b6440 0x7f7c240cf050 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 -- 172.21.15.105:0/1204159625 >> 172.21.15.105:0/1204159625 conn(0x7f7c2408bfb0 msgr2=0x7f7c240d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 -- 172.21.15.105:0/1204159625 shutdown_connections 2024-09-21T07:27:17.079 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.077+0000 7f7c2a35e640 1 -- 172.21.15.105:0/1204159625 wait complete. 2024-09-21T07:27:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.078+0000 7f7c2a35e640 1 Processor -- start 2024-09-21T07:27:17.080 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.079+0000 7f7c2a35e640 1 -- start start 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 0x7f7c24152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 0x7f7c240b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240cfc50 con 0x7f7c2408c3c0 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240cfad0 con 0x7f7c240b6850 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c2a35e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7c24162c70 con 0x7f7c240b6440 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58384/0 (socket says 172.21.15.105:58384) 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 -- 172.21.15.105:0/4055629327 learned_addr learned my addr 172.21.15.105:0/4055629327 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c237fe640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 0x7f7c24152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.083 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c288fb640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 0x7f7c240b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 -- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 msgr2=0x7f7c24152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 0x7f7c24152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 -- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 msgr2=0x7f7c240b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 0x7f7c240b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 -- 172.21.15.105:0/4055629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c241f6a50 con 0x7f7c2408c3c0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c288fb640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 0x7f7c240b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c237fe640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 0x7f7c24152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.080+0000 7f7c23fff640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c3c0 tx=0x7f7c1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.081+0000 7f7c20ff9640 1 -- 172.21.15.105:0/4055629327 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14017070 con 0x7f7c2408c3c0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.081+0000 7f7c20ff9640 1 -- 172.21.15.105:0/4055629327 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c140035f0 con 0x7f7c2408c3c0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.081+0000 7f7c20ff9640 1 -- 172.21.15.105:0/4055629327 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14003a40 con 0x7f7c2408c3c0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.081+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c240cbbb0 con 0x7f7c2408c3c0 2024-09-21T07:27:17.084 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.081+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c240cc170 con 0x7f7c2408c3c0 2024-09-21T07:27:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.083+0000 7f7c20ff9640 1 -- 172.21.15.105:0/4055629327 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c1401a070 con 0x7f7c2408c3c0 2024-09-21T07:27:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.083+0000 7f7c20ff9640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 0x7f7c1007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.083+0000 7f7c20ff9640 1 -- 172.21.15.105:0/4055629327 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7c1409a190 con 0x7f7c2408c3c0 2024-09-21T07:27:17.085 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.083+0000 7f7c237fe640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 0x7f7c1007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.084+0000 7f7c237fe640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 0x7f7c1007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c18034000 tx=0x7f7c18004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.084+0000 7f7c312cd740 1 Processor -- start 2024-09-21T07:27:17.086 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.084+0000 7f7c312cd740 1 -- start start 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422122b270 0x5642215fcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x56422161a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x564221545820 con 0x5642212023b0 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564221534c50 con 0x56422120a590 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x564221538a60 con 0x56422122b270 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c237fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58392/0 (socket says 172.21.15.105:58392) 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c237fe640 1 -- 172.21.15.105:0/1748034269 learned_addr learned my addr 172.21.15.105:0/1748034269 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c23fff640 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422122b270 0x5642215fcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.085+0000 7f7c288fb640 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x56422161a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 -- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422122b270 msgr2=0x5642215fcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422122b270 0x5642215fcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 -- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 msgr2=0x56422161a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x56422161a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 -- 172.21.15.105:0/1748034269 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564221296850 con 0x5642212023b0 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c237fe640 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c1802aac0 tx=0x7f7c1803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee50edee4d433de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c09ffb640 1 -- 172.21.15.105:0/1748034269 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1803c420 con 0x5642212023b0 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c09ffb640 1 -- 172.21.15.105:0/1748034269 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c1803c6b0 con 0x5642212023b0 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.086+0000 7f7c09ffb640 1 -- 172.21.15.105:0/1748034269 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1803cac0 con 0x5642212023b0 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 -- 172.21.15.105:0/1748034269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 msgr2=0x5642215fd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c1802aac0 tx=0x7f7c1803e070 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 -- 172.21.15.105:0/1748034269 shutdown_connections 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x56422161a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5642212023b0 0x5642215fd4e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 --2- 172.21.15.105:0/1748034269 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422122b270 0x5642215fcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 -- 172.21.15.105:0/1748034269 >> 172.21.15.105:0/1748034269 conn(0x56422121f3c0 msgr2=0x5642215d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 -- 172.21.15.105:0/1748034269 shutdown_connections 2024-09-21T07:27:17.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.087+0000 7f7c312cd740 1 -- 172.21.15.105:0/1748034269 wait complete. 2024-09-21T07:27:17.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.088+0000 7f7c312cd740 1 -- start start 2024-09-21T07:27:17.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.088+0000 7f7c312cd740 1 Processor -- start 2024-09-21T07:27:17.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642215f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x564221610350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.094 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x5642212a5620 con 0x56422122b270 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642214da5f0 con 0x5642212023b0 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x5642215fda20 con 0x56422120a590 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c312cd740 10 client.0 _ll_register_callbacks cb 0x5642214de0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58400/0 (socket says 172.21.15.105:58400) 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c288fb640 1 -- 172.21.15.105:0/2748991809 learned_addr learned my addr 172.21.15.105:0/2748991809 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c23fff640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642215f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.089+0000 7f7c237fe640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x564221610350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 -- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 msgr2=0x564221610350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x564221610350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 -- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 msgr2=0x5642215f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642215f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.095 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 -- 172.21.15.105:0/2748991809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56422130d6b0 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ce20 tx=0x7f7c1c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546c33b545be3029 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c01c060 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7c1c009ba0 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1803c6b0 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c022050 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c312cd740 10 client.5226 fetch_fsmap learned FSMap version 18 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5642215fa290 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c1803c6b0 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.090+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f7c1c025020 con 0x56422122b270 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 10 client.5226 fetch_fsmap finished waiting for FSMap version 18 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 10 client.5226 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 20 client.5226 populate_metadata read hostname 'smithi105' 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x5642211e9e40 0x5642215eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.096 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 4 client.5226 mds_command: new command op to 5073 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 --> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564221547770 con 0x5642211e9e40 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c23fff640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x5642211e9e40 0x5642215eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c02ffd640 10 client.5226 ms_handle_connect on v2:172.21.15.105:6832/1752443650 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.091+0000 7f7c23fff640 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x5642211e9e40 0x5642215eafc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c5f1cfedf56440f server_cookie=66f2d7f306b57716 in_seq=0 out_seq=0 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c02ffd640 1 -- 172.21.15.105:0/2748991809 <== mds.0 v2:172.21.15.105:6832/1752443650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564221547770 con 0x5642211e9e40 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c02ffd640 10 client.5226 handle_command_reply: tid=0 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 client.5226 shutdown 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 msgr2=0x56422162a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c00ce20 tx=0x7f7c1c007680 comp rx=0 tx=0).stop 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 shutdown_connections 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x5642211e9e40 0x5642215eafc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x56422162a580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c02ffd640 0 client.5226 ms_handle_reset on v2:172.21.15.105:6832/1752443650 2024-09-21T07:27:17.097 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x564221610350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2748991809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642215f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 >> 172.21.15.105:0/2748991809 conn(0x56422121f3c0 msgr2=0x5642215ec760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 shutdown_connections 2024-09-21T07:27:17.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 1 -- 172.21.15.105:0/2748991809 wait complete. 2024-09-21T07:27:17.098 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.092+0000 7f7c312cd740 20 client.5226 trim_cache size 0 max 16384 2024-09-21T07:27:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.110+0000 7f7c312cd740 1 Processor -- start 2024-09-21T07:27:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.110+0000 7f7c312cd740 1 -- start start 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642216315c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f7c242017c0 con 0x56422122b270 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7c240ce590 con 0x5642212023b0 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.111+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f7c241f9df0 con 0x56422120a590 2024-09-21T07:27:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58402/0 (socket says 172.21.15.105:58402) 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 -- 172.21.15.105:0/2136529382 learned_addr learned my addr 172.21.15.105:0/2136529382 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c23fff640 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642216315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 -- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 msgr2=0x5642215f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 -- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 msgr2=0x5642216315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642216315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 -- 172.21.15.105:0/2136529382 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642215f5930 con 0x56422122b270 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c23fff640 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642216315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c288fb640 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c017440 tx=0x7f7c1c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc7e931f41fb541 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c097fa640 1 -- 172.21.15.105:0/2136529382 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c01c020 con 0x56422122b270 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c097fa640 1 -- 172.21.15.105:0/2136529382 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c1c009cb0 con 0x56422122b270 2024-09-21T07:27:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.112+0000 7f7c097fa640 1 -- 172.21.15.105:0/2136529382 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c009ea0 con 0x56422122b270 2024-09-21T07:27:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 -- 172.21.15.105:0/2136529382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 msgr2=0x5642215f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c017440 tx=0x7f7c1c009900 comp rx=0 tx=0).stop 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 -- 172.21.15.105:0/2136529382 shutdown_connections 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 --2- 172.21.15.105:0/2136529382 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642212023b0 0x5642216315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 -- 172.21.15.105:0/2136529382 >> 172.21.15.105:0/2136529382 conn(0x56422121f3c0 msgr2=0x5642215d37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 -- 172.21.15.105:0/2136529382 shutdown_connections 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.113+0000 7f7c312cd740 1 -- 172.21.15.105:0/2136529382 wait complete. 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.114+0000 7f7c312cd740 1 -- start start 2024-09-21T07:27:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.114+0000 7f7c312cd740 1 Processor -- start 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642212023b0 0x564221691710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x56422162b2b0 con 0x56422122b270 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642215f64c0 con 0x56422120a590 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x564221610b90 con 0x5642212023b0 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c312cd740 10 client.0 _ll_register_callbacks cb 0x7f7c1c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c288fb640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58406/0 (socket says 172.21.15.105:58406) 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.115+0000 7f7c288fb640 1 -- 172.21.15.105:0/3289157461 learned_addr learned my addr 172.21.15.105:0/3289157461 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c23fff640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642212023b0 0x564221691710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c237fe640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 -- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642212023b0 msgr2=0x564221691710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642212023b0 0x564221691710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 -- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 msgr2=0x5642215f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 -- 172.21.15.105:0/3289157461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564221697830 con 0x56422122b270 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c288fb640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c004af0 tx=0x7f7c1c009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd996697cddb4634 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c036020 con 0x56422122b270 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mon.0 v2:172.21.15.105:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f7c1c00c070 con 0x56422122b270 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1c009cb0 con 0x56422122b270 2024-09-21T07:27:17.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f7c1c03d050 con 0x56422122b270 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c312cd740 10 client.5232 fetch_fsmap learned FSMap version 18 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5642215eb750 con 0x56422122b270 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mon.0 v2:172.21.15.105:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c1c009cb0 con 0x56422122b270 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.116+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mon.0 v2:172.21.15.105:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f7c1c040020 con 0x56422122b270 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 10 client.5232 fetch_fsmap finished waiting for FSMap version 18 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 10 client.5232 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 20 client.5232 populate_metadata read hostname 'smithi105' 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x56422169b700 0x564221628ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 4 client.5232 mds_command: new command op to 5073 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 --> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] -- command(tid 0: {"prefix": "status"}) -- 0x5642215faa50 con 0x56422169b700 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c23fff640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x56422169b700 0x564221628ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c02ffd640 10 client.5232 ms_handle_connect on v2:172.21.15.105:6832/1752443650 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c23fff640 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x56422169b700 0x564221628ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c74143717e63f02f server_cookie=ab3a5d425c9ab98d in_seq=0 out_seq=0 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c02ffd640 1 -- 172.21.15.105:0/3289157461 <== mds.0 v2:172.21.15.105:6832/1752443650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5642215faa50 con 0x56422169b700 2024-09-21T07:27:17.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c02ffd640 10 client.5232 handle_command_reply: tid=0 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.117+0000 7f7c312cd740 1 client.5232 shutdown 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 msgr2=0x5642215f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c004af0 tx=0x7f7c1c009890 comp rx=0 tx=0).stop 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 shutdown_connections 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:6832/1752443650,v1:172.21.15.105:6833/1752443650] conn(0x56422169b700 0x564221628ea0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56422120a590 0x5642215f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56422122b270 0x5642215f8970 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c02ffd640 0 client.5232 ms_handle_reset on v2:172.21.15.105:6832/1752443650 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 --2- 172.21.15.105:0/3289157461 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x5642212023b0 0x564221691710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 >> 172.21.15.105:0/3289157461 conn(0x56422121f3c0 msgr2=0x5642216315c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 shutdown_connections 2024-09-21T07:27:17.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 1 -- 172.21.15.105:0/3289157461 wait complete. 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c312cd740 20 client.5232 trim_cache size 0 max 16384 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "cluster_fsid": "15ee7004-a3e2-41ce-b1eb-6358972a1ae2", 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "whoami": 0, 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "id": 5073, 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "want_state": "up:active", 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "state": "up:active", 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "fs_name": "cephfs", 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "rank_uptime": 5.7199520369999997, 2024-09-21T07:27:17.125 INFO:teuthology.orchestra.run.smithi105.stdout: "mdsmap_epoch": 18, 2024-09-21T07:27:17.126 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch": 41, 2024-09-21T07:27:17.126 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_epoch_barrier": 41, 2024-09-21T07:27:17.126 INFO:teuthology.orchestra.run.smithi105.stdout: "uptime": 12.615725527 2024-09-21T07:27:17.126 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 msgr2=0x7f7c1007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 0x7f7c1007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c18034000 tx=0x7f7c18004730 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 msgr2=0x7f7c241f6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c3c0 tx=0x7f7c1400c880 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 shutdown_connections 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f7c10078b90 0x7f7c1007b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7c240b6850 0x7f7c240b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7c240b6440 0x7f7c24152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 --2- 172.21.15.105:0/4055629327 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7c2408c3c0 0x7f7c241f6510 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 >> 172.21.15.105:0/4055629327 conn(0x7f7c2408bfb0 msgr2=0x7f7c240b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 shutdown_connections 2024-09-21T07:27:17.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.118+0000 7f7c2a35e640 1 -- 172.21.15.105:0/4055629327 wait complete. 2024-09-21T07:27:17.131 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:27:17.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 -- 172.21.15.105:0/963826619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e54004d40 msgr2=0x7f8e54005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 --2- 172.21.15.105:0/963826619 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e54004d40 0x7f8e54005150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009ed0 tx=0x7f8e54005830 comp rx=0 tx=0).stop 2024-09-21T07:27:17.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e70fb3640 1 -- 172.21.15.105:0/963826619 reap_dead start 2024-09-21T07:27:17.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 -- 172.21.15.105:0/963826619 shutdown_connections 2024-09-21T07:27:17.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 -- 172.21.15.105:0/963826619 >> 172.21.15.105:0/963826619 conn(0x7f8e6c09d850 msgr2=0x7f8e6c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 -- 172.21.15.105:0/963826619 shutdown_connections 2024-09-21T07:27:17.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.242+0000 7f8e72a16640 1 -- 172.21.15.105:0/963826619 wait complete. 2024-09-21T07:27:17.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.244+0000 7f8e72a16640 1 Processor -- start 2024-09-21T07:27:17.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.244+0000 7f8e72a16640 1 -- start start 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e54004d40 0x7f8e6c15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8e6c1f9220 0x7f8e6c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c06f250 con 0x7f8e6c1f8680 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6c0d3b30 con 0x7f8e54004d40 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8e6c06f3d0 con 0x7f8e6c1f9220 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58424/0 (socket says 172.21.15.105:58424) 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 -- 172.21.15.105:0/882980902 learned_addr learned my addr 172.21.15.105:0/882980902 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e6bfff640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e54004d40 0x7f8e6c15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e6b7fe640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8e6c1f9220 0x7f8e6c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 -- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8e6c1f9220 msgr2=0x7f8e6c15e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.249 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8e6c1f9220 0x7f8e6c15e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 -- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e54004d40 msgr2=0x7f8e6c15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e54004d40 0x7f8e6c15fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 -- 172.21.15.105:0/882980902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6c1f8fd0 con 0x7f8e6c1f8680 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e70fb3640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402af20 tx=0x7f8e64033dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64042070 con 0x7f8e6c1f8680 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e64003c60 con 0x7f8e6c1f8680 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e6c0cac70 con 0x7f8e6c1f8680 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64038040 con 0x7f8e6c1f8680 2024-09-21T07:27:17.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.246+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e6c0c9550 con 0x7f8e6c1f8680 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.247+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e6c0caa60 con 0x7f8e6c1f8680 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.248+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e64004290 con 0x7f8e6c1f8680 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.249+0000 7f8e68ff9640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.249+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f8e640bb530 con 0x7f8e6c1f8680 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.249+0000 7f8e6bfff640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:17.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.249+0000 7f8e6bfff640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005da0 tx=0x7f8e5c00b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:17.255 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.253+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e6c1f8fd0 con 0x7f8e6c1f8680 2024-09-21T07:27:17.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.470+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8e6c0cef60 con 0x7f8e6c1f8680 2024-09-21T07:27:17.472 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.471+0000 7f8e68ff9640 1 -- 172.21.15.105:0/882980902 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f8e64083190 con 0x7f8e6c1f8680 2024-09-21T07:27:17.473 INFO:teuthology.orchestra.run.smithi105.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 msgr2=0x7f8e5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 0x7f8e5807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c005da0 tx=0x7f8e5c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 msgr2=0x7f8e6c1f8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8e6402af20 tx=0x7f8e64033dc0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 shutdown_connections 2024-09-21T07:27:17.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f8e58078b90 0x7f8e5807b0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e6c1f8680 0x7f8e6c1f8a90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8e6c1f9220 0x7f8e6c15e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 --2- 172.21.15.105:0/882980902 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e54004d40 0x7f8e6c15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 >> 172.21.15.105:0/882980902 conn(0x7f8e6c09d850 msgr2=0x7f8e6c1ea3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 shutdown_connections 2024-09-21T07:27:17.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17.473+0000 7f8e72a16640 1 -- 172.21.15.105:0/882980902 wait complete. 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi105.front.sepia.ceph.com 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:27:17.486 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:27:17.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:27:17.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:27:17.487 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:27:17.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.487 DEBUG:teuthology.orchestra.run.smithi105:> ip addr 2024-09-21T07:27:17.502 INFO:teuthology.orchestra.run.smithi105.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 ::1/128 scope host 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b8 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f0 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:d8:a4:b9 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: altname enp7s0f1 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:27:17.503 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:28 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether 0c:c4:7a:8f:d4:29 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: inet 172.21.15.105/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft 42736sec preferred_lft 42736sec 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link noprefixroute 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether be:9e:60:ae:39:48 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::b417:e7ff:febd:7b35/64 scope link 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:27:17.504 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether ee:f5:9f:7b:e4:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::ecf5:9fff:fe7b:e4c5/64 scope link 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout: link/ether be:9e:60:ae:39:48 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout: inet6 fe80::bc9e:60ff:feae:3948/64 scope link 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run.smithi105.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:27:17.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.505 DEBUG:teuthology.orchestra.run.smithi105:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:27:17.566 INFO:teuthology.orchestra.run.smithi105.stdout:1 2024-09-21T07:27:17.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.568 DEBUG:teuthology.orchestra.run.smithi105:> route 2024-09-21T07:27:17.624 INFO:teuthology.orchestra.run.smithi105.stdout:Kernel IP routing table 2024-09-21T07:27:17.624 INFO:teuthology.orchestra.run.smithi105.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:27:17.624 INFO:teuthology.orchestra.run.smithi105.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:27:17.624 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:27:17.625 INFO:teuthology.orchestra.run.smithi105.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> set -e 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:27:17.625 DEBUG:teuthology.orchestra.run.smithi105:> ') 2024-09-21T07:27:17.713 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:27:17.744 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:27:17.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.750 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:27:17.766 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:27:17.766 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:27:17.766 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:27:17.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.766 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link set brx.0 up 2024-09-21T07:27:17.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:27:17.831 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:27:17.886 INFO:teuthology.orchestra.run.smithi105.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:27:17.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:27:17.886 DEBUG:teuthology.orchestra.run.smithi105:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:27:17.942 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-21T07:27:17.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:17.942 DEBUG:teuthology.orchestra.run.smithi105:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:27:18.036 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:27:18.037 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,nofallback 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel". 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: resolved to: "172.21.15.105:3300,172.21.15.112:3300,172.21.15.112:3301" 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: trying mount with new device syntax: 0@15ee7004-a3e2-41ce-b1eb-6358972a1ae2.cephfs=/ 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.105:3300/172.21.15.112:3300/172.21.15.112:3301" will pass to kernel 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:27:18.049 INFO:teuthology.orchestra.run.smithi105.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:27:18.050 INFO:teuthology.orchestra.run.smithi105.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:27:18.050 INFO:teuthology.orchestra.run.smithi105.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:27:18.050 INFO:teuthology.orchestra.run.smithi105.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:27:18.051 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:27:18.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:18.051 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:27:18.077 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:27:18.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:27:18.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:27:18.177 INFO:teuthology.orchestra.run.smithi105.stdout:client5044 2024-09-21T07:27:18.177 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:18.177 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:18.193 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:27:18.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 -- 172.21.15.105:0/2493537052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c0afc30 msgr2=0x7f990c0b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 --2- 172.21.15.105:0/2493537052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c0afc30 0x7f990c0b08d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f990c072f40 tx=0x7f990002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 -- 172.21.15.105:0/2493537052 shutdown_connections 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 --2- 172.21.15.105:0/2493537052 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c0afc30 0x7f990c0b08d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 --2- 172.21.15.105:0/2493537052 >> v2:172.21.15.112:3300/0 conn(0x7f990c15b6d0 0x7f990c0af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 -- 172.21.15.105:0/2493537052 >> 172.21.15.105:0/2493537052 conn(0x7f990c0b6670 msgr2=0x7f990c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 -- 172.21.15.105:0/2493537052 shutdown_connections 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.343+0000 7f99128cc640 1 -- 172.21.15.105:0/2493537052 wait complete. 2024-09-21T07:27:18.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.345+0000 7f99128cc640 1 Processor -- start 2024-09-21T07:27:18.347 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.345+0000 7f99128cc640 1 -- start start 2024-09-21T07:27:18.349 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f990c0afc30 0x7f990c06cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f990c0b0e10 0x7f990c13d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f990c15bc60 con 0x7f990c15b6d0 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f990c15bae0 con 0x7f990c0afc30 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f990c15fb30 con 0x7f990c0b0e10 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58444/0 (socket says 172.21.15.105:58444) 2024-09-21T07:27:18.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 -- 172.21.15.105:0/531788879 learned_addr learned my addr 172.21.15.105:0/531788879 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:18.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f990bfff640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f990c0afc30 0x7f990c06cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f990b7fe640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f990c0b0e10 0x7f990c13d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 -- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f990c0b0e10 msgr2=0x7f990c13d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f990c0b0e10 0x7f990c13d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 -- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f990c0afc30 msgr2=0x7f990c06cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f990c0afc30 0x7f990c06cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 -- 172.21.15.105:0/531788879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c0b22a0 con 0x7f990c15b6d0 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9910e69640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f990400c760 tx=0x7f990400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904017070 con 0x7f990c15b6d0 2024-09-21T07:27:18.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f990400ce10 con 0x7f990c15b6d0 2024-09-21T07:27:18.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c0b14e0 con 0x7f990c15b6d0 2024-09-21T07:27:18.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040052b0 con 0x7f990c15b6d0 2024-09-21T07:27:18.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.347+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c0b1aa0 con 0x7f990c15b6d0 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.349+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c1f7bb0 con 0x7f990c15b6d0 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.349+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f990401a070 con 0x7f990c15b6d0 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.349+0000 7f9908ff9640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 0x7f98f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.350+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f990409a360 con 0x7f990c15b6d0 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.350+0000 7f990bfff640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 0x7f98f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.350+0000 7f990bfff640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 0x7f98f807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc00a250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:18.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.354+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9904062050 con 0x7f990c15b6d0 2024-09-21T07:27:18.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.558+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f990c1f7e60 con 0x7f98f8078b90 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.561+0000 7f9908ff9640 1 -- 172.21.15.105:0/531788879 <== mgr.4111 v2:172.21.15.112:6800/1954024229 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f990c1f7e60 con 0x7f98f8078b90 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stdout: { 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stdout: "name": "cephfs" 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-09-21T07:27:18.562 INFO:teuthology.orchestra.run.smithi105.stdout:] 2024-09-21T07:27:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 msgr2=0x7f98f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 0x7f98f807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005fd0 tx=0x7f98fc00a250 comp rx=0 tx=0).stop 2024-09-21T07:27:18.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 msgr2=0x7f990c13dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f990400c760 tx=0x7f990400cc20 comp rx=0 tx=0).stop 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 shutdown_connections 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f98f8078b90 0x7f98f807b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f990c15b6d0 0x7f990c13dc30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f990c0b0e10 0x7f990c13d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 --2- 172.21.15.105:0/531788879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f990c0afc30 0x7f990c06cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 >> 172.21.15.105:0/531788879 conn(0x7f990c0b6670 msgr2=0x7f990c0b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 shutdown_connections 2024-09-21T07:27:18.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.562+0000 7f99128cc640 1 -- 172.21.15.105:0/531788879 wait complete. 2024-09-21T07:27:18.576 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T07:27:18.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 -- 172.21.15.105:0/779871174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00c59c0 msgr2=0x7f57a00b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.693 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 --2- 172.21.15.105:0/779871174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00c59c0 0x7f57a00b2230 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f57a00746d0 tx=0x7f579802a820 comp rx=0 tx=0).stop 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 -- 172.21.15.105:0/779871174 shutdown_connections 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 --2- 172.21.15.105:0/779871174 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a00c59c0 0x7f57a00b2230 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 --2- 172.21.15.105:0/779871174 >> v2:172.21.15.112:3300/0 conn(0x7f57a009ba30 0x7f57a00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 -- 172.21.15.105:0/779871174 >> 172.21.15.105:0/779871174 conn(0x7f57a00cedb0 msgr2=0x7f57a00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 -- 172.21.15.105:0/779871174 shutdown_connections 2024-09-21T07:27:18.694 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.691+0000 7f57a4807640 1 -- 172.21.15.105:0/779871174 wait complete. 2024-09-21T07:27:18.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.693+0000 7f57a4807640 1 Processor -- start 2024-09-21T07:27:18.695 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.693+0000 7f57a4807640 1 -- start start 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 0x7f57a015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 0x7f57a01f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b28f0 con 0x7f57a009ba30 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57a00b2770 con 0x7f57a00c59c0 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f57a4807640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f57a00b2a70 con 0x7f57a00d0620 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f579e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f579e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58458/0 (socket says 172.21.15.105:58458) 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.694+0000 7f579e59c640 1 -- 172.21.15.105:0/1933682158 learned_addr learned my addr 172.21.15.105:0/1933682158 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:18.698 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f5797fff640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 0x7f57a015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 -- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 msgr2=0x7f57a01f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579ed9d640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 0x7f57a01f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 0x7f57a01f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 -- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 msgr2=0x7f57a015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 0x7f57a015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 -- 172.21.15.105:0/1933682158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a0164cb0 con 0x7f57a009ba30 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f5797fff640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 0x7f57a015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579ed9d640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 0x7f57a01f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f579e59c640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f579000d6d0 tx=0x7f579000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790017070 con 0x7f57a009ba30 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5790005330 con 0x7f57a009ba30 2024-09-21T07:27:18.699 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a015b730 con 0x7f57a009ba30 2024-09-21T07:27:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790009820 con 0x7f57a009ba30 2024-09-21T07:27:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.695+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a015bcf0 con 0x7f57a009ba30 2024-09-21T07:27:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.696+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a00d00e0 con 0x7f57a009ba30 2024-09-21T07:27:18.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.697+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57900099c0 con 0x7f57a009ba30 2024-09-21T07:27:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.697+0000 7f5796ffd640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 0x7f578007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.698+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f579009af90 con 0x7f57a009ba30 2024-09-21T07:27:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.698+0000 7f5797fff640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 0x7f578007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:18.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.698+0000 7f5797fff640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 0x7f578007b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a0164600 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:18.704 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.702+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57900632d0 con 0x7f57a009ba30 2024-09-21T07:27:18.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:18.937+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f57a00b3c00 con 0x7f57a009ba30 2024-09-21T07:27:19.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.430+0000 7f5796ffd640 1 -- 172.21.15.105:0/1933682158 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) ==== 92+0+0 (secure 0 0 0) 0x7f5790062bd0 con 0x7f57a009ba30 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.434+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 msgr2=0x7f578007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.434+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 0x7f578007b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f57a0164600 tx=0x7f5788009040 comp rx=0 tx=0).stop 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.434+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 msgr2=0x7f57a01f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.434+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f579000d6d0 tx=0x7f579000db90 comp rx=0 tx=0).stop 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 shutdown_connections 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5780078b90 0x7f578007b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a00c59c0 0x7f57a015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57a00d0620 0x7f57a01f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 --2- 172.21.15.105:0/1933682158 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f57a009ba30 0x7f57a01f70c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 >> 172.21.15.105:0/1933682158 conn(0x7f57a00cedb0 msgr2=0x7f57a00cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.435+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 shutdown_connections 2024-09-21T07:27:19.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.436+0000 7f57a4807640 1 -- 172.21.15.105:0/1933682158 wait complete. 2024-09-21T07:27:19.451 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 -- 172.21.15.105:0/2886184074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200b1a90 msgr2=0x7fcb200b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 --2- 172.21.15.105:0/2886184074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200b1a90 0x7fcb200b1ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb14005fd0 tx=0x7fcb1402a540 comp rx=0 tx=0).stop 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 -- 172.21.15.105:0/2886184074 shutdown_connections 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 --2- 172.21.15.105:0/2886184074 >> v2:172.21.15.112:3300/0 conn(0x7fcb200ce300 0x7fcb200ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 --2- 172.21.15.105:0/2886184074 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200b1a90 0x7fcb200b1ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 -- 172.21.15.105:0/2886184074 >> 172.21.15.105:0/2886184074 conn(0x7fcb200cee40 msgr2=0x7fcb200cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 -- 172.21.15.105:0/2886184074 shutdown_connections 2024-09-21T07:27:19.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.579+0000 7fcb267f9640 1 -- 172.21.15.105:0/2886184074 wait complete. 2024-09-21T07:27:19.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.580+0000 7fcb267f9640 1 Processor -- start 2024-09-21T07:27:19.583 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.581+0000 7fcb267f9640 1 -- start start 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb200b1a90 0x7fcb200cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 0x7fcb2015fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 0x7fcb2015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fcb200ccfb0 con 0x7fcb200c5a50 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2009bc40 con 0x7fcb200b1a90 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb267f9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fcb200cd130 con 0x7fcb200ce300 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 0x7fcb2015fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 0x7fcb2015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb1ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb200b1a90 0x7fcb200cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.586 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 0x7fcb2015fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58464/0 (socket says 172.21.15.105:58464) 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb1f7fe640 1 -- 172.21.15.105:0/3468855444 learned_addr learned my addr 172.21.15.105:0/3468855444 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 -- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb200b1a90 msgr2=0x7fcb200cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb200b1a90 0x7fcb200cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 -- 172.21.15.105:0/3468855444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 msgr2=0x7fcb2015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 0x7fcb2015fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.582+0000 7fcb24d96640 1 -- 172.21.15.105:0/3468855444 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb200b71d0 con 0x7fcb200ce300 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb24d96640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 0x7fcb2015ff40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800eb80 tx=0x7fcb1800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18003b70 con 0x7fcb200ce300 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb18003d10 con 0x7fcb200ce300 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb200b5390 con 0x7fcb200ce300 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb18004280 con 0x7fcb200ce300 2024-09-21T07:27:19.587 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.583+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb200b5950 con 0x7fcb200ce300 2024-09-21T07:27:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.584+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb200c94c0 con 0x7fcb200ce300 2024-09-21T07:27:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.585+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 6) ==== 101458+0+0 (secure 0 0 0) 0x7fcb18004420 con 0x7fcb200ce300 2024-09-21T07:27:19.588 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.586+0000 7fcb1cff9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 0x7fcb0c07b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.586+0000 7fcb1ffff640 1 -- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 msgr2=0x7fcb0c07b120 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.586+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fcb1809b9c0 con 0x7fcb200ce300 2024-09-21T07:27:19.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.586+0000 7fcb1ffff640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 0x7fcb0c07b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:19.593 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.591+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb1809de20 con 0x7fcb200ce300 2024-09-21T07:27:19.779 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.778+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fcb200b1ea0 con 0x7fcb200ce300 2024-09-21T07:27:19.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.784+0000 7fcb1cff9640 1 -- 172.21.15.105:0/3468855444 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v33)=0 v33) ==== 138+0+0 (secure 0 0 0) 0x7fcb18063620 con 0x7fcb200ce300 2024-09-21T07:27:19.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 msgr2=0x7fcb0c07b120 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:19.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 0x7fcb0c07b120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 msgr2=0x7fcb2015ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 0x7fcb2015ff40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcb1800eb80 tx=0x7fcb1800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 shutdown_connections 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7fcb0c078c10 0x7fcb0c07b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcb200ce300 0x7fcb2015ff40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcb200c5a50 0x7fcb2015fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 --2- 172.21.15.105:0/3468855444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb200b1a90 0x7fcb200cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 >> 172.21.15.105:0/3468855444 conn(0x7fcb200cee40 msgr2=0x7fcb200b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 shutdown_connections 2024-09-21T07:27:19.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.786+0000 7fcb267f9640 1 -- 172.21.15.105:0/3468855444 wait complete. 2024-09-21T07:27:19.800 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T07:27:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 -- 172.21.15.105:0/771737740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0c5a50 msgr2=0x7f548c0d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 --2- 172.21.15.105:0/771737740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0c5a50 0x7f548c0d0170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f54840098e0 tx=0x7f548402a530 comp rx=0 tx=0).stop 2024-09-21T07:27:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 -- 172.21.15.105:0/771737740 shutdown_connections 2024-09-21T07:27:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 --2- 172.21.15.105:0/771737740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 0x7f548c0b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.919 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 --2- 172.21.15.105:0/771737740 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0c5a50 0x7f548c0d0170 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 -- 172.21.15.105:0/771737740 >> 172.21.15.105:0/771737740 conn(0x7f548c0cee40 msgr2=0x7f548c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 -- 172.21.15.105:0/771737740 shutdown_connections 2024-09-21T07:27:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.916+0000 7f5493407640 1 -- 172.21.15.105:0/771737740 wait complete. 2024-09-21T07:27:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.918+0000 7f5493407640 1 Processor -- start 2024-09-21T07:27:19.920 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.918+0000 7f5493407640 1 -- start start 2024-09-21T07:27:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.919+0000 7f5493407640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.923 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.919+0000 7f5493407640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f548c0c5a50 0x7f548c1f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.919+0000 7f5493407640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 0x7f548c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5493407640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f548c0ccfb0 con 0x7f548c0b1fd0 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5493407640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f548c0cec60 con 0x7f548c0c5a50 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5493407640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f548c0cd130 con 0x7f548c0d06b0 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58484/0 (socket says 172.21.15.105:58484) 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 -- 172.21.15.105:0/2765464167 learned_addr learned my addr 172.21.15.105:0/2765464167 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:19.924 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54909a2640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f548c0c5a50 0x7f548c1f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54919a4640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 0x7f548c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 msgr2=0x7f548c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 0x7f548c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f548c0c5a50 msgr2=0x7f548c1f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f548c0c5a50 0x7f548c1f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 -- 172.21.15.105:0/2765464167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c06f930 con 0x7f548c0b1fd0 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f54911a3640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f548402ab20 tx=0x7f54840052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548403f070 con 0x7f548c0b1fd0 2024-09-21T07:27:19.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5484004250 con 0x7f548c0b1fd0 2024-09-21T07:27:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c06fc10 con 0x7f548c0b1fd0 2024-09-21T07:27:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.920+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484005cf0 con 0x7f548c0b1fd0 2024-09-21T07:27:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.921+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c1f6cf0 con 0x7f548c0b1fd0 2024-09-21T07:27:19.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.922+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c06a5b0 con 0x7f548c0b1fd0 2024-09-21T07:27:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.922+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 101458+0+0 (secure 0 0 0) 0x7f5484042070 con 0x7f548c0b1fd0 2024-09-21T07:27:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.923+0000 7f5489ffb640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 0x7f547807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.923+0000 7f54909a2640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 msgr2=0x7f547807b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.923+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f54840bbd20 con 0x7f548c0b1fd0 2024-09-21T07:27:19.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.923+0000 7f54909a2640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 0x7f547807b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:19.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:19.927+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f548c06a5b0 con 0x7f548c0b1fd0 2024-09-21T07:27:20.126 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.124+0000 7f54909a2640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 msgr2=0x7f547807b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:20.127 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.124+0000 7f54909a2640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 0x7f547807b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:20.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.148+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f548c0d0170 con 0x7f548c0b1fd0 2024-09-21T07:27:20.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.151+0000 7f5489ffb640 1 -- 172.21.15.105:0/2765464167 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f5484083980 con 0x7f548c0b1fd0 2024-09-21T07:27:20.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 msgr2=0x7f547807b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 0x7f547807b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 msgr2=0x7f548c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f548402ab20 tx=0x7f54840052f0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 shutdown_connections 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f5478078be0 0x7f547807b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f548c0c5a50 0x7f548c1f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f548c0d06b0 0x7f548c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 --2- 172.21.15.105:0/2765464167 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f548c0b1fd0 0x7f548c06e8c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.153+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 >> 172.21.15.105:0/2765464167 conn(0x7f548c0cee40 msgr2=0x7f548c15c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.154+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 shutdown_connections 2024-09-21T07:27:20.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.154+0000 7f5493407640 1 -- 172.21.15.105:0/2765464167 wait complete. 2024-09-21T07:27:20.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:20.167 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-09-21T07:27:20.189 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 -- 172.21.15.105:0/275350206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 msgr2=0x7f0e240af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 --2- 172.21.15.105:0/275350206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240af6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009830 tx=0x7f0e1402a510 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 -- 172.21.15.105:0/275350206 shutdown_connections 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 --2- 172.21.15.105:0/275350206 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e240b34c0 0x7f0e2415f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 --2- 172.21.15.105:0/275350206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240af6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.347+0000 7f0e28d72640 1 -- 172.21.15.105:0/275350206 >> 172.21.15.105:0/275350206 conn(0x7f0e240b6610 msgr2=0x7f0e240b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.348+0000 7f0e28d72640 1 -- 172.21.15.105:0/275350206 shutdown_connections 2024-09-21T07:27:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.348+0000 7f0e28d72640 1 -- 172.21.15.105:0/275350206 wait complete. 2024-09-21T07:27:20.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.349+0000 7f0e28d72640 1 Processor -- start 2024-09-21T07:27:20.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.349+0000 7f0e28d72640 1 -- start start 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 0x7f0e24157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e2415f700 0x7f0e240cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0e241560c0 con 0x7f0e240b34c0 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e24155f40 con 0x7f0e2415cd90 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e28d72640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0e240b4d30 con 0x7f0e2415f700 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 0x7f0e24157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 0x7f0e24157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38634/0 (socket says 172.21.15.105:38634) 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e21d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 -- 172.21.15.105:0/2787926814 learned_addr learned my addr 172.21.15.105:0/2787926814 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:20.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e22d9d640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e2415f700 0x7f0e240cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e2415f700 msgr2=0x7f0e240cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e2415f700 0x7f0e240cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 msgr2=0x7f0e240b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e241f3bf0 con 0x7f0e240b34c0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e2415cd90 0x7f0e240b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.351+0000 7f0e2259c640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 0x7f0e24157b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009950 tx=0x7f0e14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.352+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14005c90 con 0x7f0e240b34c0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.352+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e14005310 con 0x7f0e240b34c0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.352+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e241f7f00 con 0x7f0e240b34c0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.352+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e140047a0 con 0x7f0e240b34c0 2024-09-21T07:27:20.355 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.352+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e241f4180 con 0x7f0e240b34c0 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.353+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e2415b940 con 0x7f0e240b34c0 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.353+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 6) ==== 101458+0+0 (secure 0 0 0) 0x7f0e14004940 con 0x7f0e240b34c0 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.354+0000 7f0e1affd640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.354+0000 7f0e21d9b640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 msgr2=0x7f0e0407b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.354+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0e140bc180 con 0x7f0e240b34c0 2024-09-21T07:27:20.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.354+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:20.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.363+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e2415b940 con 0x7f0e240b34c0 2024-09-21T07:27:20.557 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.556+0000 7f0e21d9b640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 msgr2=0x7f0e0407b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:20.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.556+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:20.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.570+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0e241f7cb0 con 0x7f0e04078be0 2024-09-21T07:27:20.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.957+0000 7f0e21d9b640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 msgr2=0x7f0e0407b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:20.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:20.957+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-21T07:27:21.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:21.758+0000 7f0e21d9b640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 msgr2=0x7f0e0407b0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1954024229 2024-09-21T07:27:21.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:21.758+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-21T07:27:22.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:22.923+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 7) ==== 101169+0+0 (secure 0 0 0) 0x7f0e1407ffd0 con 0x7f0e240b34c0 2024-09-21T07:27:22.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:22.923+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 msgr2=0x7f0e0407b0f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:22.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:22.923+0000 7f0e1affd640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1954024229,v1:172.21.15.112:6801/1954024229] conn(0x7f0e04078be0 0x7f0e0407b0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:23.101 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca90222640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca90222640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca90222640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca90222640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca90222640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca8d21c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca8d21c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca8d21c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca8d21c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.102 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:23.099+0000 7fca8d21c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:23.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:23.928+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mgrmap(e 8) ==== 101296+0+0 (secure 0 0 0) 0x7f0e14004c30 con 0x7f0e240b34c0 2024-09-21T07:27:23.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:23.928+0000 7f0e1affd640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0e04081120 0x7f0e04083540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:23.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:23.928+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 --> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0e140aa170 con 0x7f0e04081120 2024-09-21T07:27:23.931 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:23.929+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0e04081120 0x7f0e04083540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:23.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:23.932+0000 7f0e21d9b640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0e04081120 0x7f0e04083540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e241ee200 tx=0x7f0e0c012040 comp rx=0 tx=0).ready entity=mgr.4832 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.106 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.104+0000 7f0e1affd640 1 -- 172.21.15.105:0/2787926814 <== mgr.4832 v2:172.21.15.112:6800/1472023028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e140aa170 con 0x7f0e04081120 2024-09-21T07:27:24.106 INFO:teuthology.orchestra.run.smithi105.stdout:Schedule set for path snap_test_dir1 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0e04081120 msgr2=0x7f0e04083540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0e04081120 0x7f0e04083540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0e241ee200 tx=0x7f0e0c012040 comp rx=0 tx=0).stop 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 msgr2=0x7f0e24157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 --2- 172.21.15.105:0/2787926814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0e240b34c0 0x7f0e24157b80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009950 tx=0x7f0e14004270 comp rx=0 tx=0).stop 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e22d9d640 1 -- 172.21.15.105:0/2787926814 reap_dead start 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 shutdown_connections 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 >> 172.21.15.105:0/2787926814 conn(0x7f0e240b6610 msgr2=0x7f0e2415d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 shutdown_connections 2024-09-21T07:27:24.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.107+0000 7f0e28d72640 1 -- 172.21.15.105:0/2787926814 wait complete. 2024-09-21T07:27:24.121 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-09-21T07:27:24.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2024-09-21T07:27:24.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 -- 172.21.15.105:0/253470213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 msgr2=0x7f8c4c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 --2- 172.21.15.105:0/253470213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c06d4b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009830 tx=0x7f8c4002a510 comp rx=0 tx=0).stop 2024-09-21T07:27:24.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 -- 172.21.15.105:0/253470213 shutdown_connections 2024-09-21T07:27:24.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 --2- 172.21.15.105:0/253470213 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c0ae5e0 0x7f8c4c0ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 --2- 172.21.15.105:0/253470213 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c06d4b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 -- 172.21.15.105:0/253470213 >> 172.21.15.105:0/253470213 conn(0x7f8c4c0b65a0 msgr2=0x7f8c4c0b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 -- 172.21.15.105:0/253470213 shutdown_connections 2024-09-21T07:27:24.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.237+0000 7f8c5239b640 1 -- 172.21.15.105:0/253470213 wait complete. 2024-09-21T07:27:24.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.239+0000 7f8c5239b640 1 Processor -- start 2024-09-21T07:27:24.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.239+0000 7f8c5239b640 1 -- start start 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.240+0000 7f8c5239b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4c0ae5e0 0x7f8c4c162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.240+0000 7f8c5239b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c15f980 0x7f8c4c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c0af0b0 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c0aef30 con 0x7f8c4c0ae5e0 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c0af230 con 0x7f8c4c15f980 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38644/0 (socket says 172.21.15.105:38644) 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 -- 172.21.15.105:0/3016474682 learned_addr learned my addr 172.21.15.105:0/3016474682 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c50938640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c15f980 0x7f8c4c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4bfff640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4c0ae5e0 0x7f8c4c162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 -- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c15f980 msgr2=0x7f8c4c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.244 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c15f980 0x7f8c4c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 -- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4c0ae5e0 msgr2=0x7f8c4c162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4c0ae5e0 0x7f8c4c162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 -- 172.21.15.105:0/3016474682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c163370 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c4b7fe640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009950 tx=0x7f8c40033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40031d60 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c40033e10 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c1611c0 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40005260 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.245 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.241+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c161780 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.242+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4c0af730 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.243+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 101296+0+0 (secure 0 0 0) 0x7f8c40005400 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.244+0000 7f8c48ff9640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 0x7f8c3807af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.244+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f8c400bbdd0 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.244+0000 7f8c4bfff640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 0x7f8c3807af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.246 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.244+0000 7f8c4bfff640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 0x7f8c3807af40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c000f60 tx=0x7f8c3c009040 comp rx=0 tx=0).ready entity=mgr.4832 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.250 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.248+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c4003f030 con 0x7f8c4c15e8e0 2024-09-21T07:27:24.452 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.450+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 --> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8c4c160f70 con 0x7f8c38078a30 2024-09-21T07:27:24.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.514+0000 7f8c48ff9640 1 -- 172.21.15.105:0/3016474682 <== mgr.4832 v2:172.21.15.112:6800/1472023028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+39 (secure 0 0 0) 0x7f8c4c160f70 con 0x7f8c38078a30 2024-09-21T07:27:24.516 INFO:teuthology.orchestra.run.smithi105.stdout:Retention added to path snap_test_dir1 2024-09-21T07:27:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 msgr2=0x7f8c3807af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 0x7f8c3807af40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c000f60 tx=0x7f8c3c009040 comp rx=0 tx=0).stop 2024-09-21T07:27:24.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 msgr2=0x7f8c4c1598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009950 tx=0x7f8c40033c20 comp rx=0 tx=0).stop 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 shutdown_connections 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f8c38078a30 0x7f8c3807af40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8c4c15f980 0x7f8c4c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c4c15e8e0 0x7f8c4c1598e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 --2- 172.21.15.105:0/3016474682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8c4c0ae5e0 0x7f8c4c162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 >> 172.21.15.105:0/3016474682 conn(0x7f8c4c0b65a0 msgr2=0x7f8c4c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 shutdown_connections 2024-09-21T07:27:24.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.516+0000 7f8c5239b640 1 -- 172.21.15.105:0/3016474682 wait complete. 2024-09-21T07:27:24.529 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-09-21T07:27:24.530 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.customsnapkernel/scheduled-2024-09-21-07_28_00 in ~36s... 2024-09-21T07:27:24.530 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2024-09-21T07:27:24.530 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-09-21T07:27:24.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 -- 172.21.15.105:0/2785171228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd9e0 msgr2=0x7f98400b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 --2- 172.21.15.105:0/2785171228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd9e0 0x7f98400b1b00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9840074940 tx=0x7f983802aa30 comp rx=0 tx=0).stop 2024-09-21T07:27:24.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 -- 172.21.15.105:0/2785171228 shutdown_connections 2024-09-21T07:27:24.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 --2- 172.21.15.105:0/2785171228 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd9e0 0x7f98400b1b00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 --2- 172.21.15.105:0/2785171228 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cd090 0x7f98400cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 --2- 172.21.15.105:0/2785171228 >> v2:172.21.15.112:3300/0 conn(0x7f98400cff70 0x7f98400d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 -- 172.21.15.105:0/2785171228 >> 172.21.15.105:0/2785171228 conn(0x7f98400cee20 msgr2=0x7f98400cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 -- 172.21.15.105:0/2785171228 shutdown_connections 2024-09-21T07:27:24.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.642+0000 7f98451ad640 1 -- 172.21.15.105:0/2785171228 wait complete. 2024-09-21T07:27:24.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.644+0000 7f98451ad640 1 Processor -- start 2024-09-21T07:27:24.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.644+0000 7f98451ad640 1 -- start start 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98400cd9e0 0x7f98401e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 0x7f984009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f98400b7950 con 0x7f98400cd090 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f98400b77d0 con 0x7f98400cd9e0 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f98400cddf0 con 0x7f98400cff70 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38658/0 (socket says 172.21.15.105:38658) 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 -- 172.21.15.105:0/1743840681 learned_addr learned my addr 172.21.15.105:0/1743840681 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983659c640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98400cd9e0 0x7f98401e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983f59e640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 0x7f984009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 -- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 msgr2=0x7f984009c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 0x7f984009c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 -- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98400cd9e0 msgr2=0x7f98401e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98400cd9e0 0x7f98401e6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 -- 172.21.15.105:0/1743840681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984009dd90 con 0x7f98400cd090 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983f59e640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 0x7f984009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f983ed9d640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f983000e7c0 tx=0x7f983000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000ccc0 con 0x7f98400cd090 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9830010430 con 0x7f98400cd090 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984009e070 con 0x7f98400cd090 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.646+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830010860 con 0x7f98400cd090 2024-09-21T07:27:24.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.647+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401e6dc0 con 0x7f98400cd090 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.647+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984009bc80 con 0x7f98400cd090 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.648+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 8) ==== 101296+0+0 (secure 0 0 0) 0x7f9830010a00 con 0x7f98400cd090 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.649+0000 7f9837fff640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 0x7f982007af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.649+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f983009b630 con 0x7f98400cd090 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.649+0000 7f983659c640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 0x7f982007af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.649+0000 7f983659c640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 0x7f982007af70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98401e5770 tx=0x7f9828005f90 comp rx=0 tx=0).ready entity=mgr.4832 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.653+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f984009bc80 con 0x7f98400cd090 2024-09-21T07:27:24.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.850+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 --> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f984009db40 con 0x7f9820078a60 2024-09-21T07:27:24.858 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.857+0000 7f9837fff640 1 -- 172.21.15.105:0/1743840681 <== mgr.4832 v2:172.21.15.112:6800/1472023028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7f984009db40 con 0x7f9820078a60 2024-09-21T07:27:24.858 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:24.859 INFO:teuthology.orchestra.run.smithi105.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 msgr2=0x7f982007af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 0x7f982007af70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98401e5770 tx=0x7f9828005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 msgr2=0x7f984009eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f983000e7c0 tx=0x7f983000ec80 comp rx=0 tx=0).stop 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 shutdown_connections 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f9820078a60 0x7f982007af70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98400cd9e0 0x7f98401e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.861 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98400cff70 0x7f984009c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 --2- 172.21.15.105:0/1743840681 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f98400cd090 0x7f984009eb50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 >> 172.21.15.105:0/1743840681 conn(0x7f98400cee20 msgr2=0x7f98401e4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 shutdown_connections 2024-09-21T07:27:24.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.859+0000 7f98451ad640 1 -- 172.21.15.105:0/1743840681 wait complete. 2024-09-21T07:27:24.872 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-09-21T07:27:24.872 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2024-09-21T07:27:24.872 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 -- 172.21.15.105:0/2488013592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f4005280 msgr2=0x7f84f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 --2- 172.21.15.105:0/2488013592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f4005280 0x7f84f400b9c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8504001f00 tx=0x7f84f4005830 comp rx=0 tx=0).stop 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 -- 172.21.15.105:0/2488013592 shutdown_connections 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 --2- 172.21.15.105:0/2488013592 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84f4005280 0x7f84f400b9c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 -- 172.21.15.105:0/2488013592 >> 172.21.15.105:0/2488013592 conn(0x7f850c0cdce0 msgr2=0x7f850c0ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 -- 172.21.15.105:0/2488013592 shutdown_connections 2024-09-21T07:27:24.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.990+0000 7f85111da640 1 -- 172.21.15.105:0/2488013592 wait complete. 2024-09-21T07:27:24.993 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.992+0000 7f85111da640 1 Processor -- start 2024-09-21T07:27:24.994 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.992+0000 7f85111da640 1 -- start start 2024-09-21T07:27:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 0x7f850c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f850c06d7e0 0x7f850c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f850c15c8b0 con 0x7f850c09df60 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f850c15c730 con 0x7f84f4005280 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f85111da640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f850c15ca30 con 0x7f850c06d7e0 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f850b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f850a59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f850c06d7e0 0x7f850c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f850b59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38674/0 (socket says 172.21.15.105:38674) 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f850b59e640 1 -- 172.21.15.105:0/3739881596 learned_addr learned my addr 172.21.15.105:0/3739881596 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.993+0000 7f850ad9d640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 0x7f850c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 -- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f850c06d7e0 msgr2=0x7f850c1584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f850c06d7e0 0x7f850c1584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 -- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 msgr2=0x7f850c14e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:24.997 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 0x7f850c14e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 -- 172.21.15.105:0/3739881596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c09cee0 con 0x7f850c09df60 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850ad9d640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 0x7f850c14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f850b59e640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f850402ada0 tx=0x7f8504001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504003d80 con 0x7f850c09df60 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8504032470 con 0x7f850c09df60 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c09d1c0 con 0x7f850c09df60 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040328a0 con 0x7f850c09df60 2024-09-21T07:27:24.998 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.994+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c09d780 con 0x7f850c09df60 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.995+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c09ca00 con 0x7f850c09df60 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.996+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 101561+0+0 (secure 0 0 0) 0x7f850403d030 con 0x7f850c09df60 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.996+0000 7f84f37fe640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 0x7f84f807b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.997+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f85040baf20 con 0x7f850c09df60 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.997+0000 7f850ad9d640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 0x7f84f807b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:24.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:24.997+0000 7f850ad9d640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 0x7f84f807b210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009460 tx=0x7f84fc005f90 comp rx=0 tx=0).ready entity=mgr.4832 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:25.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.001+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f850c09ca00 con 0x7f850c09df60 2024-09-21T07:27:25.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.200+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 --> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f850c0b3220 con 0x7f84f8078d00 2024-09-21T07:27:25.280 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.278+0000 7f84f37fe640 1 -- 172.21.15.105:0/3739881596 <== mgr.4832 v2:172.21.15.112:6800/1472023028 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f850c0b3220 con 0x7f84f8078d00 2024-09-21T07:27:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:Schedule removed for path snap_test_dir1 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 msgr2=0x7f84f807b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 0x7f84f807b210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009460 tx=0x7f84fc005f90 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 msgr2=0x7f850c158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f850402ada0 tx=0x7f8504001bf0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 shutdown_connections 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f84f8078d00 0x7f84f807b210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f850c09df60 0x7f850c158a10 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f850c06d7e0 0x7f850c1584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 --2- 172.21.15.105:0/3739881596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84f4005280 0x7f850c14e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 >> 172.21.15.105:0/3739881596 conn(0x7f850c0cdce0 msgr2=0x7f850c074380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 shutdown_connections 2024-09-21T07:27:25.283 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.281+0000 7f85111da640 1 -- 172.21.15.105:0/3739881596 wait complete. 2024-09-21T07:27:25.293 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-09-21T07:27:25.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:25.294 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.customsnapkernel) 2024-09-21T07:27:25.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:25.310 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-09-21T07:27:25.368 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-09-21T07:27:25.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 -- 172.21.15.105:0/1410019775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 msgr2=0x7f284c06f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.522 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 --2- 172.21.15.105:0/1410019775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c06f040 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f283c009820 tx=0x7f283c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 -- 172.21.15.105:0/1410019775 shutdown_connections 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 --2- 172.21.15.105:0/1410019775 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0b9790 0x7f284c0d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 --2- 172.21.15.105:0/1410019775 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c06f040 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 -- 172.21.15.105:0/1410019775 >> 172.21.15.105:0/1410019775 conn(0x7f284c09d850 msgr2=0x7f284c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 -- 172.21.15.105:0/1410019775 shutdown_connections 2024-09-21T07:27:25.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.520+0000 7f28510d5640 1 -- 172.21.15.105:0/1410019775 wait complete. 2024-09-21T07:27:25.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.522+0000 7f28510d5640 1 Processor -- start 2024-09-21T07:27:25.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.522+0000 7f28510d5640 1 -- start start 2024-09-21T07:27:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c1f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:25.526 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0d3880 0x7f284c0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0d0e70 con 0x7f284c0b9790 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f284c0d0cf0 con 0x7f284c0b3710 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f284c0d0ff0 con 0x7f284c0d3880 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38684/0 (socket says 172.21.15.105:38684) 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c1f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 -- 172.21.15.105:0/4035191824 learned_addr learned my addr 172.21.15.105:0/4035191824 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284b59e640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0d3880 0x7f284c0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 -- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0d3880 msgr2=0x7f284c0ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0d3880 0x7f284c0ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 -- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 msgr2=0x7f284c1f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c1f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 -- 172.21.15.105:0/4035191824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c0cd200 con 0x7f284c0b9790 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f284259c640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f283400e9f0 tx=0x7f283400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:25.527 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400cd80 con 0x7f284c0b9790 2024-09-21T07:27:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2834004020 con 0x7f284c0b9790 2024-09-21T07:27:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c0cd4e0 con 0x7f284c0b9790 2024-09-21T07:27:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.524+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834004470 con 0x7f284c0b9790 2024-09-21T07:27:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.525+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c0cdaa0 con 0x7f284c0b9790 2024-09-21T07:27:25.528 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.525+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f284c0d1510 con 0x7f284c0b9790 2024-09-21T07:27:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.526+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 9) ==== 101561+0+0 (secure 0 0 0) 0x7f28340035f0 con 0x7f284c0b9790 2024-09-21T07:27:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.527+0000 7f2843fff640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 0x7f282c07b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.527+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f283409bd50 con 0x7f284c0b9790 2024-09-21T07:27:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.527+0000 7f284ad9d640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 0x7f282c07b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:25.529 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.527+0000 7f284ad9d640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 0x7f282c07b210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f283c034000 tx=0x7f283c004730 comp rx=0 tx=0).ready entity=mgr.4832 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:25.533 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.531+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f283409f030 con 0x7f284c0b9790 2024-09-21T07:27:25.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.755+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f284c06e600 con 0x7f284c0b9790 2024-09-21T07:27:25.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.980+0000 7f2843fff640 1 -- 172.21.15.105:0/4035191824 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v10) ==== 93+0+0 (secure 0 0 0) 0x7f2834063610 con 0x7f284c0b9790 2024-09-21T07:27:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.982+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 msgr2=0x7f282c07b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.983+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 0x7f282c07b210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f283c034000 tx=0x7f283c004730 comp rx=0 tx=0).stop 2024-09-21T07:27:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.983+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 msgr2=0x7f284c0b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.983+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f283400e9f0 tx=0x7f283400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 shutdown_connections 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f282c078d00 0x7f282c07b210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f284c0d3880 0x7f284c0ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f284c0b9790 0x7f284c0b4cc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 --2- 172.21.15.105:0/4035191824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f284c0b3710 0x7f284c1f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 >> 172.21.15.105:0/4035191824 conn(0x7f284c09d850 msgr2=0x7f284c068e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 shutdown_connections 2024-09-21T07:27:25.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:25.984+0000 7f28510d5640 1 -- 172.21.15.105:0/4035191824 wait complete. 2024-09-21T07:27:25.998 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:27:25.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:27:26.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:27:26.087 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:27:26.087 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:27:26.088 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:26.088 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:26.103 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:26.104 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:26.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:27:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 -- 172.21.15.105:0/1965171798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200ae090 msgr2=0x7f46200ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 --2- 172.21.15.105:0/1965171798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200ae090 0x7f46200ae4a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f462006d250 tx=0x7f460802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 -- 172.21.15.105:0/1965171798 shutdown_connections 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 --2- 172.21.15.105:0/1965171798 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200ae090 0x7f46200ae4a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 --2- 172.21.15.105:0/1965171798 >> v2:172.21.15.112:3300/0 conn(0x7f462015a3a0 0x7f462006d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 -- 172.21.15.105:0/1965171798 >> 172.21.15.105:0/1965171798 conn(0x7f46200b6610 msgr2=0x7f46200b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 -- 172.21.15.105:0/1965171798 shutdown_connections 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.313+0000 7f4625bb0640 1 -- 172.21.15.105:0/1965171798 wait complete. 2024-09-21T07:27:26.316 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.315+0000 7f4625bb0640 1 Processor -- start 2024-09-21T07:27:26.317 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.315+0000 7f4625bb0640 1 -- start start 2024-09-21T07:27:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200d06e0 0x7f46201591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.319 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462015a3a0 0x7f4620144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f462015e200 con 0x7f46200ae090 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f462015e080 con 0x7f462015a3a0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4625bb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f462015e380 con 0x7f46200d06e0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f461f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38710/0 (socket says 172.21.15.105:38710) 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f461f7fe640 1 -- 172.21.15.105:0/2148183395 learned_addr learned my addr 172.21.15.105:0/2148183395 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f4617fff640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200d06e0 0x7f46201591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.316+0000 7f461ffff640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462015a3a0 0x7f4620144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 -- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200d06e0 msgr2=0x7f46201591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200d06e0 0x7f46201591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 -- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462015a3a0 msgr2=0x7f4620144e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462015a3a0 0x7f4620144e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 -- 172.21.15.105:0/2148183395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620159730 con 0x7f46200ae090 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461f7fe640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f461000e9f0 tx=0x7f461000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:26.320 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000cd80 con 0x7f46200ae090 2024-09-21T07:27:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46201453d0 con 0x7f46200ae090 2024-09-21T07:27:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4610004020 con 0x7f46200ae090 2024-09-21T07:27:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610004470 con 0x7f46200ae090 2024-09-21T07:27:26.321 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.317+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4620157c30 con 0x7f46200ae090 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.318+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462015e830 con 0x7f46200ae090 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.319+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f46100035f0 con 0x7f46200ae090 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.319+0000 7f461cff9640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 0x7f460007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.320+0000 7f4617fff640 1 -- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 msgr2=0x7f460007b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.320+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f461009bb70 con 0x7f46200ae090 2024-09-21T07:27:26.322 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.320+0000 7f4617fff640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 0x7f460007b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:26.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.324+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4610063430 con 0x7f46200ae090 2024-09-21T07:27:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.518+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46200cd7a0 con 0x7f46200ae090 2024-09-21T07:27:26.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.519+0000 7f461cff9640 1 -- 172.21.15.105:0/2148183395 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v42) ==== 80+0+360 (secure 0 0 0) 0x7f4610067e60 con 0x7f46200ae090 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:0/2921736159 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:0/3749496995 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:0/1978324892 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:0/1737949316 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:6801/1954024229 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stdout:172.21.15.112:6800/1954024229 2024-09-22T07:27:22.915542+0000 2024-09-21T07:27:26.521 INFO:teuthology.orchestra.run.smithi105.stderr:listed 6 entries 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 msgr2=0x7f460007b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 0x7f460007b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 msgr2=0x7f4620157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f461000e9f0 tx=0x7f461000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 shutdown_connections 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f4600078ce0 0x7f460007b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f462015a3a0 0x7f4620144e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f46200d06e0 0x7f46201591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 --2- 172.21.15.105:0/2148183395 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46200ae090 0x7f4620157320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.523 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 >> 172.21.15.105:0/2148183395 conn(0x7f46200b6610 msgr2=0x7f462015c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 shutdown_connections 2024-09-21T07:27:26.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.521+0000 7f4625bb0640 1 -- 172.21.15.105:0/2148183395 wait complete. 2024-09-21T07:27:26.534 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:27:26.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:26.534 DEBUG:teuthology.orchestra.run.smithi105:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:27:26.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:27:26.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:26.568 DEBUG:teuthology.orchestra.run.smithi105:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:27:26.626 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:27:26.626 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:27:26.627 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:26.642 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15681 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:27:26.642 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:27:26.642 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:26.642 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:26.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1257 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:27:26.683 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:27:26.684 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:27:26.684 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:26.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15681 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:27:26.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 -- 172.21.15.105:0/2588668586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f640cb4a0 msgr2=0x7f5f64073dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 --2- 172.21.15.105:0/2588668586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f640cb4a0 0x7f5f64073dc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5f64073370 tx=0x7f5f5802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 -- 172.21.15.105:0/2588668586 shutdown_connections 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 --2- 172.21.15.105:0/2588668586 >> v2:172.21.15.112:3300/0 conn(0x7f5f640cc1d0 0x7f5f640cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 --2- 172.21.15.105:0/2588668586 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f640cb4a0 0x7f5f64073dc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 -- 172.21.15.105:0/2588668586 >> 172.21.15.105:0/2588668586 conn(0x7f5f640ccb90 msgr2=0x7f5f640ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.835+0000 7f5f6c430640 1 -- 172.21.15.105:0/2588668586 shutdown_connections 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.836+0000 7f5f6c430640 1 -- 172.21.15.105:0/2588668586 wait complete. 2024-09-21T07:27:26.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.837+0000 7f5f6c430640 1 Processor -- start 2024-09-21T07:27:26.840 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.837+0000 7f5f6c430640 1 -- start start 2024-09-21T07:27:26.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f640ca760 0x7f5f641f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f640cb4a0 0x7f5f640c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5f640cea80 con 0x7f5f6409aa20 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f640ce900 con 0x7f5f640ca760 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.838+0000 7f5f6c430640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5f640cec00 con 0x7f5f640cb4a0 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38732/0 (socket says 172.21.15.105:38732) 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 -- 172.21.15.105:0/3107535862 learned_addr learned my addr 172.21.15.105:0/3107535862 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a1cc640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f640ca760 0x7f5f641f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f699cb640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f640cb4a0 0x7f5f640c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f640cb4a0 msgr2=0x7f5f640c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f640cb4a0 0x7f5f640c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.844 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f640ca760 msgr2=0x7f5f641f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f640ca760 0x7f5f641f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 -- 172.21.15.105:0/3107535862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f640c63e0 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6a9cd640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00ea40 tx=0x7f5f5c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00ce40 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f5c004c10 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f641f8e60 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c005020 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.839+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f641f9420 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.841+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f640cb000 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.841+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f5f5c0051c0 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.841+0000 7f5f56ffd640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 0x7f5f4807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.842+0000 7f5f6a1cc640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 msgr2=0x7f5f4807b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.842+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f5f5c09be70 con 0x7f5f6409aa20 2024-09-21T07:27:26.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.842+0000 7f5f6a1cc640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 0x7f5f4807b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:26.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:26.846+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f640cb000 con 0x7f5f6409aa20 2024-09-21T07:27:27.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.043+0000 7f5f6a1cc640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 msgr2=0x7f5f4807b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:27.045 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.043+0000 7f5f6a1cc640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 0x7f5f4807b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.068+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f640cf120 con 0x7f5f6409aa20 2024-09-21T07:27:27.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.068+0000 7f5f56ffd640 1 -- 172.21.15.105:0/3107535862 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7f5f5c063840 con 0x7f5f6409aa20 2024-09-21T07:27:27.071 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:27.071 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:27.071 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 msgr2=0x7f5f4807b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 0x7f5f4807b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 msgr2=0x7f5f641f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00ea40 tx=0x7f5f5c00ef00 comp rx=0 tx=0).stop 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 shutdown_connections 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f5f48078ce0 0x7f5f4807b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f6409aa20 0x7f5f641f58e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5f640cb4a0 0x7f5f640c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.073 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 --2- 172.21.15.105:0/3107535862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f640ca760 0x7f5f641f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 >> 172.21.15.105:0/3107535862 conn(0x7f5f640ccb90 msgr2=0x7f5f6415f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 shutdown_connections 2024-09-21T07:27:27.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.071+0000 7f5f6c430640 1 -- 172.21.15.105:0/3107535862 wait complete. 2024-09-21T07:27:27.084 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 -- 172.21.15.105:0/3417307013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac0b6900 msgr2=0x7f2dac0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 --2- 172.21.15.105:0/3417307013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac0b6900 0x7f2dac0b08b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2dac074a20 tx=0x7f2da002a560 comp rx=0 tx=0).stop 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 -- 172.21.15.105:0/3417307013 shutdown_connections 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 --2- 172.21.15.105:0/3417307013 >> v2:172.21.15.112:3300/0 conn(0x7f2dac06d2e0 0x7f2dac15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 --2- 172.21.15.105:0/3417307013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 0x7f2dac06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 --2- 172.21.15.105:0/3417307013 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac0b6900 0x7f2dac0b08b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 -- 172.21.15.105:0/3417307013 >> 172.21.15.105:0/3417307013 conn(0x7f2dac0b2d40 msgr2=0x7f2dac0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 -- 172.21.15.105:0/3417307013 shutdown_connections 2024-09-21T07:27:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.197+0000 7f2db2de4640 1 -- 172.21.15.105:0/3417307013 wait complete. 2024-09-21T07:27:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.199+0000 7f2db2de4640 1 Processor -- start 2024-09-21T07:27:27.201 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.199+0000 7f2db2de4640 1 -- start start 2024-09-21T07:27:27.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 0x7f2dac1f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 0x7f2dac09e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac15c760 con 0x7f2dac06d2e0 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2dac15c5e0 con 0x7f2dac0b6900 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.200+0000 7f2db2de4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f2dac15c8e0 con 0x7f2dac0b0df0 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38756/0 (socket says 172.21.15.105:38756) 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 -- 172.21.15.105:0/2332169306 learned_addr learned my addr 172.21.15.105:0/2332169306 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db1381640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 0x7f2dac09e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2dabfff640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 0x7f2dac1f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 msgr2=0x7f2dac1f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 0x7f2dac1f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 msgr2=0x7f2dac09e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.204 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 0x7f2dac09e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 -- 172.21.15.105:0/2332169306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dac1f7b00 con 0x7f2dac06d2e0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db1381640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 0x7f2dac09e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db0b80640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2da00099c0 tx=0x7f2da00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da003e030 con 0x7f2dac06d2e0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2da0003ca0 con 0x7f2dac06d2e0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dac1f7de0 con 0x7f2dac06d2e0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.201+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da0035040 con 0x7f2dac06d2e0 2024-09-21T07:27:27.205 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.202+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dac1f75c0 con 0x7f2dac06d2e0 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.203+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dac070720 con 0x7f2dac06d2e0 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.203+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f2da0041030 con 0x7f2dac06d2e0 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.204+0000 7f2da97fa640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 0x7f2d9007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.204+0000 7f2dabfff640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 msgr2=0x7f2d9007b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.204+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f2da00bb750 con 0x7f2dac06d2e0 2024-09-21T07:27:27.206 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.204+0000 7f2dabfff640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 0x7f2d9007b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:27.210 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.208+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2dac070720 con 0x7f2dac06d2e0 2024-09-21T07:27:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.405+0000 7f2dabfff640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 msgr2=0x7f2d9007b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:27.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.405+0000 7f2dabfff640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 0x7f2d9007b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.437+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dac0b38a0 con 0x7f2dac06d2e0 2024-09-21T07:27:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.438+0000 7f2da97fa640 1 -- 172.21.15.105:0/2332169306 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7f2dac0b38a0 con 0x7f2dac06d2e0 2024-09-21T07:27:27.440 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:27.440 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:27.440 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 msgr2=0x7f2d9007b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 0x7f2d9007b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 msgr2=0x7f2dac14f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2da00099c0 tx=0x7f2da00038a0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 shutdown_connections 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f2d90078cb0 0x7f2d9007b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2dac0b6900 0x7f2dac09e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2dac0b0df0 0x7f2dac1f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 --2- 172.21.15.105:0/2332169306 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2dac06d2e0 0x7f2dac14f1d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 >> 172.21.15.105:0/2332169306 conn(0x7f2dac0b2d40 msgr2=0x7f2dac070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 shutdown_connections 2024-09-21T07:27:27.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.440+0000 7f2db2de4640 1 -- 172.21.15.105:0/2332169306 wait complete. 2024-09-21T07:27:27.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:27.569 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 -- 172.21.15.105:0/1366964848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f0005350 msgr2=0x7ff0f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1366964848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f0005350 0x7ff0f000b9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001dc0 tx=0x7ff0f0005830 comp rx=0 tx=0).stop 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 -- 172.21.15.105:0/1366964848 shutdown_connections 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1366964848 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff0f0005350 0x7ff0f000b9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 -- 172.21.15.105:0/1366964848 >> 172.21.15.105:0/1366964848 conn(0x7ff10406cda0 msgr2=0x7ff10406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 -- 172.21.15.105:0/1366964848 shutdown_connections 2024-09-21T07:27:27.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.567+0000 7ff10bc55640 1 -- 172.21.15.105:0/1366964848 wait complete. 2024-09-21T07:27:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.569+0000 7ff10bc55640 1 Processor -- start 2024-09-21T07:27:27.571 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.569+0000 7ff10bc55640 1 -- start start 2024-09-21T07:27:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0f0005350 0x7ff1040aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 0x7ff10414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.573 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040d0a80 con 0x7ff104154410 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff1040d0900 con 0x7ff1040b1f50 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.570+0000 7ff10bc55640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7ff1040d0c00 con 0x7ff0f0005350 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38786/0 (socket says 172.21.15.105:38786) 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 -- 172.21.15.105:0/1913652709 learned_addr learned my addr 172.21.15.105:0/1913652709 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff1099f1640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0f0005350 0x7ff1040aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff1091f0640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 0x7ff10414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 -- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0f0005350 msgr2=0x7ff1040aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0f0005350 0x7ff1040aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.574 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 -- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 msgr2=0x7ff10414b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 0x7ff10414b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 -- 172.21.15.105:0/1913652709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1040b3200 con 0x7ff104154410 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff1091f0640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 0x7ff10414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10a1f2640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001eb0 tx=0x7ff0fc006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc003d80 con 0x7ff104154410 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0fc005b00 con 0x7ff104154410 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1040b2360 con 0x7ff104154410 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc005f50 con 0x7ff104154410 2024-09-21T07:27:27.575 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.571+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1040b1470 con 0x7ff104154410 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.573+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1040b2c90 con 0x7ff104154410 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.573+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7ff0fc005030 con 0x7ff104154410 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.573+0000 7ff0fa7fc640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 0x7ff0e807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.574+0000 7ff1099f1640 1 -- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 msgr2=0x7ff0e807b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.574+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7ff0fc0bacb0 con 0x7ff104154410 2024-09-21T07:27:27.576 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.574+0000 7ff1099f1640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 0x7ff0e807b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:27.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.583+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff0fc082690 con 0x7ff104154410 2024-09-21T07:27:27.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.769+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff10406fe60 con 0x7ff104154410 2024-09-21T07:27:27.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.770+0000 7ff0fa7fc640 1 -- 172.21.15.105:0/1913652709 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17457 (secure 0 0 0) 0x7ff0fc0870c0 con 0x7ff104154410 2024-09-21T07:27:27.773 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:27.773 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":42,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:22.915594+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:27:08.726593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T07:27:09.704686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/2921736159":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/3749496995":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/1978324892":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/1737949316":"2024-09-22T07:27:22.915542+0000","172.21.15.112:6801/1954024229":"2024-09-22T07:27:22.915542+0000","172.21.15.112:6800/1954024229":"2024-09-22T07:27:22.915542+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 msgr2=0x7ff0e807b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 0x7ff0e807b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 msgr2=0x7ff104154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc001eb0 tx=0x7ff0fc006ca0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 shutdown_connections 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7ff0e8078ce0 0x7ff0e807b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff1040b1f50 0x7ff10414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff104154410 0x7ff104154820 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 --2- 172.21.15.105:0/1913652709 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff0f0005350 0x7ff1040aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 >> 172.21.15.105:0/1913652709 conn(0x7ff10406cda0 msgr2=0x7ff104151090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 shutdown_connections 2024-09-21T07:27:27.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.773+0000 7ff10bc55640 1 -- 172.21.15.105:0/1913652709 wait complete. 2024-09-21T07:27:27.786 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:27:27.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 -- 172.21.15.105:0/1278536786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 msgr2=0x7fec100d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 --2- 172.21.15.105:0/1278536786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec100d00e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec100b8020 tx=0x7fec0c02a480 comp rx=0 tx=0).stop 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 -- 172.21.15.105:0/1278536786 shutdown_connections 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 --2- 172.21.15.105:0/1278536786 >> v2:172.21.15.112:3300/0 conn(0x7fec100c59c0 0x7fec100b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 --2- 172.21.15.105:0/1278536786 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec100d00e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 -- 172.21.15.105:0/1278536786 >> 172.21.15.105:0/1278536786 conn(0x7fec100cedb0 msgr2=0x7fec100cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 -- 172.21.15.105:0/1278536786 shutdown_connections 2024-09-21T07:27:27.901 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.899+0000 7fec1747e640 1 -- 172.21.15.105:0/1278536786 wait complete. 2024-09-21T07:27:27.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.900+0000 7fec1747e640 1 Processor -- start 2024-09-21T07:27:27.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.900+0000 7fec1747e640 1 -- start start 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec10155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 0x7fec100b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fec100b28f0 con 0x7fec100c59c0 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fec100b2770 con 0x7fec100d0620 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1747e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fec100b2a70 con 0x7fec1009ba30 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec14a19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec14a19640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38792/0 (socket says 172.21.15.105:38792) 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec14a19640 1 -- 172.21.15.105:0/3891398653 learned_addr learned my addr 172.21.15.105:0/3891398653 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1521a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec10155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 0x7fec100b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 msgr2=0x7fec10155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec10155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 msgr2=0x7fec100b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 -- 172.21.15.105:0/3891398653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec10156be0 con 0x7fec100d0620 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec1521a640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec10155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec14a19640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.902+0000 7fec15a1b640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 0x7fec100b5fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec100cccf0 tx=0x7fec0800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.903+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08010630 con 0x7fec100d0620 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.903+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec0800ee90 con 0x7fec100d0620 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.903+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1015e1f0 con 0x7fec100d0620 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.903+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08010a50 con 0x7fec100d0620 2024-09-21T07:27:27.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.903+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1015e7b0 con 0x7fec100d0620 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.904+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec101639c0 con 0x7fec100d0620 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.905+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7fec08010bf0 con 0x7fec100d0620 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.905+0000 7fec05ffb640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 0x7febf407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.905+0000 7fec1521a640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 msgr2=0x7febf407b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.905+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7fec0809bf80 con 0x7fec100d0620 2024-09-21T07:27:27.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.905+0000 7fec1521a640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 0x7febf407b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:27.916 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:27.914+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec0800ce20 con 0x7fec100d0620 2024-09-21T07:27:28.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.107+0000 7fec1521a640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 msgr2=0x7febf407b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:28.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.107+0000 7fec1521a640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 0x7febf407b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:28.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.137+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec100d00e0 con 0x7fec100d0620 2024-09-21T07:27:28.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.138+0000 7fec05ffb640 1 -- 172.21.15.105:0/3891398653 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7fec08063840 con 0x7fec100d0620 2024-09-21T07:27:28.140 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:28.140 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:28.140 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:28.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 msgr2=0x7febf407b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 0x7febf407b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 msgr2=0x7fec100b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 0x7fec100b5fe0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec100cccf0 tx=0x7fec0800c6a0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 shutdown_connections 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7febf4078ce0 0x7febf407b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fec100d0620 0x7fec100b5fe0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec100c59c0 0x7fec100b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 --2- 172.21.15.105:0/3891398653 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fec1009ba30 0x7fec10155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.140+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 >> 172.21.15.105:0/3891398653 conn(0x7fec100cedb0 msgr2=0x7fec101642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.141+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 shutdown_connections 2024-09-21T07:27:28.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.141+0000 7fec1747e640 1 -- 172.21.15.105:0/3891398653 wait complete. 2024-09-21T07:27:28.154 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 -- 172.21.15.105:0/963279613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88400b33f0 msgr2=0x7f884015c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 --2- 172.21.15.105:0/963279613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88400b33f0 0x7f884015c730 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8834009860 tx=0x7f883402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 -- 172.21.15.105:0/963279613 shutdown_connections 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 --2- 172.21.15.105:0/963279613 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88400b33f0 0x7f884015c730 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 -- 172.21.15.105:0/963279613 >> 172.21.15.105:0/963279613 conn(0x7f88400cdce0 msgr2=0x7f88400ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 -- 172.21.15.105:0/963279613 shutdown_connections 2024-09-21T07:27:28.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.267+0000 7f884657b640 1 -- 172.21.15.105:0/963279613 wait complete. 2024-09-21T07:27:28.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.268+0000 7f884657b640 1 Processor -- start 2024-09-21T07:27:28.270 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.269+0000 7f884657b640 1 -- start start 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 0x7f88400af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 0x7f88400b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8840074c10 con 0x7f88400ce870 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8840074a90 con 0x7f8840073f70 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f884657b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8840074d90 con 0x7f8840159380 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38798/0 (socket says 172.21.15.105:38798) 2024-09-21T07:27:28.273 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 -- 172.21.15.105:0/1265225635 learned_addr learned my addr 172.21.15.105:0/1265225635 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f8844b18640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 0x7f88400b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883ffff640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 0x7f88400af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 msgr2=0x7f88400b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 0x7f88400b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 msgr2=0x7f88400af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.270+0000 7f883f7fe640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 0x7f88400af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883f7fe640 1 -- 172.21.15.105:0/1265225635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88401f58e0 con 0x7f88400ce870 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883ffff640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 0x7f88400af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f8844b18640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 0x7f88400b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883f7fe640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f88340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88340044e0 con 0x7f88400ce870 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8834005330 con 0x7f88400ce870 2024-09-21T07:27:28.274 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88401f5ba0 con 0x7f88400ce870 2024-09-21T07:27:28.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8834003800 con 0x7f88400ce870 2024-09-21T07:27:28.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.271+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401f6160 con 0x7f88400ce870 2024-09-21T07:27:28.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.272+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884015c730 con 0x7f88400ce870 2024-09-21T07:27:28.275 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.273+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f88340039a0 con 0x7f88400ce870 2024-09-21T07:27:28.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.273+0000 7f883cff9640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 0x7f882c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.273+0000 7f883ffff640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 msgr2=0x7f882c07b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:28.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.273+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f88340bc6d0 con 0x7f88400ce870 2024-09-21T07:27:28.276 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.273+0000 7f883ffff640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 0x7f882c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:28.279 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.277+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f883403f030 con 0x7f88400ce870 2024-09-21T07:27:28.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.474+0000 7f883ffff640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 msgr2=0x7f882c07b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:28.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.474+0000 7f883ffff640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 0x7f882c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:28.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.500+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88400c4aa0 con 0x7f88400ce870 2024-09-21T07:27:28.502 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.501+0000 7f883cff9640 1 -- 172.21.15.105:0/1265225635 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5235 (secure 0 0 0) 0x7f8834031770 con 0x7f88400ce870 2024-09-21T07:27:28.503 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:28.503 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":18,"btime":"2024-09-21T07:27:14:407866+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:27:10.743141+0000","modified":"2024-09-21T07:27:13.773777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5073},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5073":{"gid":5073,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.105:6833/1752443650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":1752443650},{"type":"v1","addr":"172.21.15.105:6833","nonce":1752443650}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5073,"qdb_cluster":[5073]},"id":2}]} 2024-09-21T07:27:28.503 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 18 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 msgr2=0x7f882c07b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 0x7f882c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 msgr2=0x7f88400b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8834000c00 tx=0x7f88340042f0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 shutdown_connections 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f882c078ce0 0x7f882c07b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8840159380 0x7f88400b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f88400ce870 0x7f88400b22e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 --2- 172.21.15.105:0/1265225635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8840073f70 0x7f88400af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 >> 172.21.15.105:0/1265225635 conn(0x7f88400cdce0 msgr2=0x7f88400709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.505 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 shutdown_connections 2024-09-21T07:27:28.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.503+0000 7f884657b640 1 -- 172.21.15.105:0/1265225635 wait complete. 2024-09-21T07:27:28.516 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:28.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 -- 172.21.15.105:0/3149206711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 msgr2=0x7f3ec40b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3149206711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b3cb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009820 tx=0x7f3eb802a510 comp rx=0 tx=0).stop 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 -- 172.21.15.105:0/3149206711 shutdown_connections 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3149206711 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b3cb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 -- 172.21.15.105:0/3149206711 >> 172.21.15.105:0/3149206711 conn(0x7f3ec408bfb0 msgr2=0x7f3ec409d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 -- 172.21.15.105:0/3149206711 shutdown_connections 2024-09-21T07:27:28.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.628+0000 7f3ecac07640 1 -- 172.21.15.105:0/3149206711 wait complete. 2024-09-21T07:27:28.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.630+0000 7f3ecac07640 1 Processor -- start 2024-09-21T07:27:28.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.630+0000 7f3ecac07640 1 -- start start 2024-09-21T07:27:28.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ec40cf1b0 0x7f3ec4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40cfcd0 con 0x7f3ec40d4460 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec40b4b30 con 0x7f3ec40d3020 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.631+0000 7f3ecac07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec40cfe50 con 0x7f3ec40cf1b0 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38802/0 (socket says 172.21.15.105:38802) 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 -- 172.21.15.105:0/3800545286 learned_addr learned my addr 172.21.15.105:0/3800545286 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec89a3640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ec40cf1b0 0x7f3ec4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec3fff640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 -- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ec40cf1b0 msgr2=0x7f3ec4156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ec40cf1b0 0x7f3ec4156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 -- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 msgr2=0x7f3ec40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 -- 172.21.15.105:0/3800545286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec41f5c60 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec91a4640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00c700 tx=0x7f3ebc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003b70 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ebc003d10 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec4156ad0 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc00aa50 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.632+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec415b920 con 0x7f3ec40d4460 2024-09-21T07:27:28.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.633+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec40d1420 con 0x7f3ec40d4460 2024-09-21T07:27:28.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.635+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f3ebc00abf0 con 0x7f3ec40d4460 2024-09-21T07:27:28.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.636+0000 7f3ec17fa640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 0x7f3ea807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.636+0000 7f3ec89a3640 1 -- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 msgr2=0x7f3ea807b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:28.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.636+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f3ebc09aef0 con 0x7f3ec40d4460 2024-09-21T07:27:28.638 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.636+0000 7f3ec89a3640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 0x7f3ea807b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:28.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.640+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ebc062830 con 0x7f3ec40d4460 2024-09-21T07:27:28.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.825+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ec406ece0 con 0x7f3ec40d4460 2024-09-21T07:27:28.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.827+0000 7f3ec17fa640 1 -- 172.21.15.105:0/3800545286 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17457 (secure 0 0 0) 0x7f3ebc067260 con 0x7f3ec40d4460 2024-09-21T07:27:28.829 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:28.829 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":42,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:22.915594+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:27:08.726593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-21T07:27:09.704686+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/2921736159":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/3749496995":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/1978324892":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/1737949316":"2024-09-22T07:27:22.915542+0000","172.21.15.112:6801/1954024229":"2024-09-22T07:27:22.915542+0000","172.21.15.112:6800/1954024229":"2024-09-22T07:27:22.915542+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 msgr2=0x7f3ea807b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 0x7f3ea807b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 msgr2=0x7f3ec41f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00c700 tx=0x7f3ebc00cbc0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 shutdown_connections 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f3ea8078cb0 0x7f3ea807b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3ec40d3020 0x7f3ec40b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3ec40d4460 0x7f3ec41f63f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 --2- 172.21.15.105:0/3800545286 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3ec40cf1b0 0x7f3ec4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 >> 172.21.15.105:0/3800545286 conn(0x7f3ec408bfb0 msgr2=0x7f3ec4068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 shutdown_connections 2024-09-21T07:27:28.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.829+0000 7f3ecac07640 1 -- 172.21.15.105:0/3800545286 wait complete. 2024-09-21T07:27:28.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:27:28.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 -- 172.21.15.105:0/934887356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980b1a90 msgr2=0x7f76980b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 --2- 172.21.15.105:0/934887356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980b1a90 0x7f76980b1ea0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7684009830 tx=0x7f768402a480 comp rx=0 tx=0).stop 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 -- 172.21.15.105:0/934887356 shutdown_connections 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 --2- 172.21.15.105:0/934887356 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980ce300 0x7f76980ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 --2- 172.21.15.105:0/934887356 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980b1a90 0x7f76980b1ea0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 -- 172.21.15.105:0/934887356 >> 172.21.15.105:0/934887356 conn(0x7f76980cee40 msgr2=0x7f76980cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 -- 172.21.15.105:0/934887356 shutdown_connections 2024-09-21T07:27:28.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.956+0000 7f769f1f1640 1 -- 172.21.15.105:0/934887356 wait complete. 2024-09-21T07:27:28.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.958+0000 7f769f1f1640 1 Processor -- start 2024-09-21T07:27:28.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.958+0000 7f769f1f1640 1 -- start start 2024-09-21T07:27:28.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980b1a90 0x7f769815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76980c5a50 0x7f769806deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f76980ccfb0 con 0x7f76980ce300 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f769809bc40 con 0x7f76980c5a50 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.959+0000 7f769f1f1640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f76980cd130 con 0x7f76980b1a90 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38820/0 (socket says 172.21.15.105:38820) 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 -- 172.21.15.105:0/3402747901 learned_addr learned my addr 172.21.15.105:0/3402747901 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769cf8d640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980b1a90 0x7f769815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f7697fff640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76980c5a50 0x7f769806deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980b1a90 msgr2=0x7f769815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980b1a90 0x7f769815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76980c5a50 msgr2=0x7f769806deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76980c5a50 0x7f769806deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 -- 172.21.15.105:0/3402747901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769806f1d0 con 0x7f76980ce300 2024-09-21T07:27:28.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769d78e640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f769000ea70 tx=0x7f769000ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769000ce40 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7690004c10 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769815c360 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690005020 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.960+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769806ec10 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.961+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76980b1ea0 con 0x7f76980ce300 2024-09-21T07:27:28.964 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.962+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f76900051c0 con 0x7f76980ce300 2024-09-21T07:27:28.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.963+0000 7f76957fa640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 0x7f767c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:28.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.963+0000 7f769cf8d640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 msgr2=0x7f767c07b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:28.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.963+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6359+0+0 (secure 0 0 0) 0x7f769009c580 con 0x7f76980ce300 2024-09-21T07:27:28.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.963+0000 7f769cf8d640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 0x7f767c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:28.969 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:28.967+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7690026020 con 0x7f76980ce300 2024-09-21T07:27:29.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.165+0000 7f769cf8d640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 msgr2=0x7f767c07b1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:29.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.165+0000 7f769cf8d640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 0x7f767c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-21T07:27:29.187 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.185+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f769806e3f0 con 0x7f76980ce300 2024-09-21T07:27:29.188 INFO:tasks.ceph.mon.a.smithi105.stderr:2024-09-21T07:27:29.186+0000 7f3c46d8f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:27:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.187+0000 7f76957fa640 1 -- 172.21.15.105:0/3402747901 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7f769806f1d0 con 0x7f76980ce300 2024-09-21T07:27:29.189 INFO:teuthology.orchestra.run.smithi105.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:27:29.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 msgr2=0x7f767c07b1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:29.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 0x7f767c07b1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.192 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 msgr2=0x7f76980b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f769000ea70 tx=0x7f769000ef30 comp rx=0 tx=0).stop 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 shutdown_connections 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f767c078ce0 0x7f767c07b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76980c5a50 0x7f769806deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f76980ce300 0x7f76980b0000 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 --2- 172.21.15.105:0/3402747901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f76980b1a90 0x7f769815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 >> 172.21.15.105:0/3402747901 conn(0x7f76980cee40 msgr2=0x7f769815c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:29.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 shutdown_connections 2024-09-21T07:27:29.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.191+0000 7f769f1f1640 1 -- 172.21.15.105:0/3402747901 wait complete. 2024-09-21T07:27:29.194 INFO:tasks.ceph.mds.a.smithi105.stderr: -3598> 2024-09-21T07:27:15.351+0000 7fc5e9af6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:27:29.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:27:29.248 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2024-09-21T07:27:29.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 -- 172.21.15.105:0/1358666545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a18004d40 msgr2=0x7f0a18005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:29.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 --2- 172.21.15.105:0/1358666545 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a18004d40 0x7f0a18005210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a24001dc0 tx=0x7f0a18005830 comp rx=0 tx=0).stop 2024-09-21T07:27:29.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a2b7fe640 1 -- 172.21.15.105:0/1358666545 reap_dead start 2024-09-21T07:27:29.328 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 -- 172.21.15.105:0/1358666545 shutdown_connections 2024-09-21T07:27:29.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 -- 172.21.15.105:0/1358666545 >> 172.21.15.105:0/1358666545 conn(0x7f0a2c0b6610 msgr2=0x7f0a2c0b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:29.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 -- 172.21.15.105:0/1358666545 shutdown_connections 2024-09-21T07:27:29.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.326+0000 7f0a31571640 1 -- 172.21.15.105:0/1358666545 wait complete. 2024-09-21T07:27:29.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.328+0000 7f0a31571640 1 Processor -- start 2024-09-21T07:27:29.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.328+0000 7f0a31571640 1 -- start start 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a18004d40 0x7f0a2c09eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 0x7f0a2c09c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c155d40 con 0x7f0a2c153f10 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c155bc0 con 0x7f0a2c09b7d0 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a31571640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c155ec0 con 0x7f0a18004d40 2024-09-21T07:27:29.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a23fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:38854/0 (socket says 172.21.15.105:38854) 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a23fff640 1 -- 172.21.15.105:0/4153844109 learned_addr learned my addr 172.21.15.105:0/4153844109 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a2affd640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a18004d40 0x7f0a2c09eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.329+0000 7f0a2b7fe640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 0x7f0a2c09c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 -- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a18004d40 msgr2=0x7f0a2c09eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a18004d40 0x7f0a2c09eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 -- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 msgr2=0x7f0a2c09c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 0x7f0a2c09c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 -- 172.21.15.105:0/4153844109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c0ce190 con 0x7f0a2c153f10 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a2b7fe640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 0x7f0a2c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a23fff640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011ae0 tx=0x7f0a14011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14004460 con 0x7f0a2c153f10 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a1400f9a0 con 0x7f0a2c153f10 2024-09-21T07:27:29.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c0d0680 con 0x7f0a2c153f10 2024-09-21T07:27:29.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1400fdd0 con 0x7f0a2c153f10 2024-09-21T07:27:29.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.330+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c09b2b0 con 0x7f0a2c153f10 2024-09-21T07:27:29.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.331+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c0b3010 con 0x7f0a2c153f10 2024-09-21T07:27:29.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.333+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 10) ==== 101544+0+0 (secure 0 0 0) 0x7f0a140035f0 con 0x7f0a2c153f10 2024-09-21T07:27:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.333+0000 7f0a237fe640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 0x7f0a0c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.333+0000 7f0a2affd640 1 -- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 msgr2=0x7f0a0c07b1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1472023028 2024-09-21T07:27:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.333+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6445+0+0 (secure 0 0 0) 0x7f0a1409c590 con 0x7f0a2c153f10 2024-09-21T07:27:29.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.333+0000 7f0a2affd640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 0x7f0a0c07b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-21T07:27:29.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.337+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a2c0b3010 con 0x7f0a2c153f10 2024-09-21T07:27:29.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.479+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 11) ==== 101152+0+0 (secure 0 0 0) 0x7f0a14060240 con 0x7f0a2c153f10 2024-09-21T07:27:29.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.479+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 msgr2=0x7f0a0c07b1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:27:29.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.479+0000 7f0a237fe640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 0x7f0a0c07b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554c064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554c064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554c064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554c064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554c064640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.531 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554905e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.532 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554905e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.532 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554905e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.532 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554905e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.532 INFO:tasks.ceph.mgr.x.smithi112.stderr:2024-09-21T07:27:29.529+0000 7f554905e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:27:29.570 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:29.568+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0a2c09b0a0 con 0x7f0a2c153f10 2024-09-21T07:27:30.191 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.189+0000 7f0a237fe640 1 -- 172.21.15.105:0/4153844109 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f0a140646d0 con 0x7f0a2c153f10 2024-09-21T07:27:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 msgr2=0x7f0a2c0cdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011ae0 tx=0x7f0a14011ee0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 shutdown_connections 2024-09-21T07:27:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:6800/1472023028,v1:172.21.15.112:6801/1472023028] conn(0x7f0a0c078cb0 0x7f0a0c07b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.193 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0a2c09b7d0 0x7f0a2c09c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0a2c153f10 0x7f0a2c0cdc50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 --2- 172.21.15.105:0/4153844109 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0a18004d40 0x7f0a2c09eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 >> 172.21.15.105:0/4153844109 conn(0x7f0a2c0b6610 msgr2=0x7f0a2c0b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 shutdown_connections 2024-09-21T07:27:30.194 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.191+0000 7f0a31571640 1 -- 172.21.15.105:0/4153844109 wait complete. 2024-09-21T07:27:30.205 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:27:30.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 -- 172.21.15.105:0/2217839342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 msgr2=0x7fa5180cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 --2- 172.21.15.105:0/2217839342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa5180cce70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa508005fd0 tx=0x7fa50802a460 comp rx=0 tx=0).stop 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 -- 172.21.15.105:0/2217839342 shutdown_connections 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 --2- 172.21.15.105:0/2217839342 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa5180cce70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 --2- 172.21.15.105:0/2217839342 >> v2:172.21.15.112:3300/0 conn(0x7fa5180c59a0 0x7fa5180d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 -- 172.21.15.105:0/2217839342 >> 172.21.15.105:0/2217839342 conn(0x7fa5180ced90 msgr2=0x7fa5180cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:30.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 -- 172.21.15.105:0/2217839342 shutdown_connections 2024-09-21T07:27:30.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.360+0000 7fa520382640 1 -- 172.21.15.105:0/2217839342 wait complete. 2024-09-21T07:27:30.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.362+0000 7fa520382640 1 Processor -- start 2024-09-21T07:27:30.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.362+0000 7fa520382640 1 -- start start 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 0x7fa5180c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 0x7fa51815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa5180b1a90 con 0x7fa5180b2f30 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa51809bc40 con 0x7fa5180c59a0 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa520382640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa5180b1c10 con 0x7fa5180b3340 2024-09-21T07:27:30.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53468/0 (socket says 172.21.15.105:53468) 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 -- 172.21.15.105:0/546858120 learned_addr learned my addr 172.21.15.105:0/546858120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51d91d640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 0x7fa5180c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e91f640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 0x7fa51815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 -- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 msgr2=0x7fa5180c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 0x7fa5180c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 -- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 msgr2=0x7fa51815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 0x7fa51815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:30.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e11e640 1 -- 172.21.15.105:0/546858120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5181f3380 con 0x7fa5180b2f30 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51d91d640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 0x7fa5180c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.363+0000 7fa51e91f640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 0x7fa51815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa51e11e640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa51400d730 tx=0x7fa51400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514017070 con 0x7fa5180b2f30 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa514009b40 con 0x7fa5180b2f30 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5181e60b0 con 0x7fa5180b2f30 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa514005020 con 0x7fa5180b2f30 2024-09-21T07:27:30.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.364+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5181f3920 con 0x7fa5180b2f30 2024-09-21T07:27:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.365+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5180d0180 con 0x7fa5180b2f30 2024-09-21T07:27:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.366+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 11) ==== 101152+0+0 (secure 0 0 0) 0x7fa5140051c0 con 0x7fa5180b2f30 2024-09-21T07:27:30.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.366+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6789+0+0 (secure 0 0 0) 0x7fa51409ac60 con 0x7fa5180b2f30 2024-09-21T07:27:30.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.371+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5140630c0 con 0x7fa5180b2f30 2024-09-21T07:27:30.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.486+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 12) ==== 101279+0+0 (secure 0 0 0) 0x7fa51405ec30 con 0x7fa5180b2f30 2024-09-21T07:27:30.489 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.487+0000 7fa50effd640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 0x7fa4fc080d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:30.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.490+0000 7fa51d91d640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 0x7fa4fc080d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:30.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.491+0000 7fa51d91d640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 0x7fa4fc080d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa508033000 tx=0x7fa508004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:30.555 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:30.554+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5181e8ae0 con 0x7fa5180b2f30 2024-09-21T07:27:31.491 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.489+0000 7fa50effd640 1 -- 172.21.15.105:0/546858120 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7fa514010460 con 0x7fa5180b2f30 2024-09-21T07:27:31.491 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.492+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 msgr2=0x7fa4fc080d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.492+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 0x7fa4fc080d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa508033000 tx=0x7fa508004730 comp rx=0 tx=0).stop 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.492+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 msgr2=0x7fa518153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa51400d730 tx=0x7fa51400dbf0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 shutdown_connections 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5180c59a0 0x7fa51815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa4fc07e930 0x7fa4fc080d50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa5180b3340 0x7fa5180c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 --2- 172.21.15.105:0/546858120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa5180b2f30 0x7fa518153730 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 >> 172.21.15.105:0/546858120 conn(0x7fa5180ced90 msgr2=0x7fa5180cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:31.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 shutdown_connections 2024-09-21T07:27:31.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.493+0000 7fa520382640 1 -- 172.21.15.105:0/546858120 wait complete. 2024-09-21T07:27:31.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:27:31.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 -- 172.21.15.105:0/3487778701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 msgr2=0x7fc1fc15b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:31.627 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 --2- 172.21.15.105:0/3487778701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc15b670 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc0cfd90 tx=0x7fc1f802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 -- 172.21.15.105:0/3487778701 shutdown_connections 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 --2- 172.21.15.105:0/3487778701 >> v2:172.21.15.112:3300/0 conn(0x7fc1fc15f180 0x7fc1fc15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 --2- 172.21.15.105:0/3487778701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc15b670 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 -- 172.21.15.105:0/3487778701 >> 172.21.15.105:0/3487778701 conn(0x7fc1fc06d3d0 msgr2=0x7fc1fc0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 -- 172.21.15.105:0/3487778701 shutdown_connections 2024-09-21T07:27:31.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.625+0000 7fc203f2b640 1 -- 172.21.15.105:0/3487778701 wait complete. 2024-09-21T07:27:31.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.627+0000 7fc203f2b640 1 Processor -- start 2024-09-21T07:27:31.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.627+0000 7fc203f2b640 1 -- start start 2024-09-21T07:27:31.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:31.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc0ce0e0 0x7fc1fc1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1fc15f180 0x7fc1fc15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc070120 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc06ffa0 con 0x7fc1fc15f180 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc203f2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fc1fc0702a0 con 0x7fc1fc0ce0e0 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.628+0000 7fc201cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53498/0 (socket says 172.21.15.105:53498) 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 -- 172.21.15.105:0/1819484502 learned_addr learned my addr 172.21.15.105:0/1819484502 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc2024c8640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1fc15f180 0x7fc1fc15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc2014c6640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc0ce0e0 0x7fc1fc1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 -- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc0ce0e0 msgr2=0x7fc1fc1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:31.632 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc0ce0e0 0x7fc1fc1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 -- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1fc15f180 msgr2=0x7fc1fc15dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1fc15f180 0x7fc1fc15dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 -- 172.21.15.105:0/1819484502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc0b0f20 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc201cc7640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1f802aea0 tx=0x7fc1f8003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8031b90 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f8031e20 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc0b1c40 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8033310 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.629+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc0b11c0 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.630+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc068d00 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.631+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 12) ==== 101279+0+0 (secure 0 0 0) 0x7fc1f80334b0 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.631+0000 7fc1f27fc640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 0x7fc1e007aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.632+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6351+0+0 (secure 0 0 0) 0x7fc1f80bc070 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.632+0000 7fc2014c6640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 0x7fc1e007aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:31.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.632+0000 7fc2014c6640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 0x7fc1e007aef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc158a80 tx=0x7fc1ec009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:31.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.636+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1fc068d00 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.816 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.814+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1fc0707c0 con 0x7fc1fc0c4fa0 2024-09-21T07:27:31.996 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:31.994+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 13) ==== 101385+0+0 (secure 0 0 0) 0x7fc1f8080220 con 0x7fc1fc0c4fa0 2024-09-21T07:27:32.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.493+0000 7fc1f27fc640 1 -- 172.21.15.105:0/1819484502 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7fc1f803f030 con 0x7fc1fc0c4fa0 2024-09-21T07:27:32.495 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' removed 2024-09-21T07:27:32.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.498+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 msgr2=0x7fc1e007aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.498+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 0x7fc1e007aef0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc158a80 tx=0x7fc1ec009040 comp rx=0 tx=0).stop 2024-09-21T07:27:32.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.498+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 msgr2=0x7fc1fc1f7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.500 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.498+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc1f802aea0 tx=0x7fc1f8003a10 comp rx=0 tx=0).stop 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 shutdown_connections 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1fc15f180 0x7fc1fc15dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fc1e00789e0 0x7fc1e007aef0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1fc0ce0e0 0x7fc1fc1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 --2- 172.21.15.105:0/1819484502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1fc0c4fa0 0x7fc1fc1f7a00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 >> 172.21.15.105:0/1819484502 conn(0x7fc1fc06d3d0 msgr2=0x7fc1fc0cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 shutdown_connections 2024-09-21T07:27:32.501 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.499+0000 7fc203f2b640 1 -- 172.21.15.105:0/1819484502 wait complete. 2024-09-21T07:27:32.514 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:27:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 -- 172.21.15.105:0/2409420862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 msgr2=0x7f2e3c0d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2409420862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c0d4870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009830 tx=0x7f2e2402a480 comp rx=0 tx=0).stop 2024-09-21T07:27:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 -- 172.21.15.105:0/2409420862 shutdown_connections 2024-09-21T07:27:32.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2409420862 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0cf1b0 0x7f2e3c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2409420862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c0d4870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 -- 172.21.15.105:0/2409420862 >> 172.21.15.105:0/2409420862 conn(0x7f2e3c08bfb0 msgr2=0x7f2e3c09d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 -- 172.21.15.105:0/2409420862 shutdown_connections 2024-09-21T07:27:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.627+0000 7f2e418c5640 1 -- 172.21.15.105:0/2409420862 wait complete. 2024-09-21T07:27:32.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.629+0000 7f2e418c5640 1 Processor -- start 2024-09-21T07:27:32.631 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.629+0000 7f2e418c5640 1 -- start start 2024-09-21T07:27:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0d3020 0x7f2e3c1f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c1f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0cfb00 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.633 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b4b30 con 0x7f2e3c0d4460 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e418c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0cfc80 con 0x7f2e3c0d3020 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.630+0000 7f2e3affd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53500/0 (socket says 172.21.15.105:53500) 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 -- 172.21.15.105:0/2579799503 learned_addr learned my addr 172.21.15.105:0/2579799503 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e327fc640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0d3020 0x7f2e3c1f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3b7fe640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c1f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 -- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0d3020 msgr2=0x7f2e3c1f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0d3020 0x7f2e3c1f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 -- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 msgr2=0x7f2e3c1f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c1f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 -- 172.21.15.105:0/2579799503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c15e7d0 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e3affd640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c400 tx=0x7f2e2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c011790 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.634 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e2c017070 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c0b4080 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c011cc0 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.635 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.631+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c1f92a0 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.632+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c0d0170 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.633+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 101385+0+0 (secure 0 0 0) 0x7f2e2c01c460 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.633+0000 7f2e38ff9640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 0x7f2e1c07b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.633+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f2e2c09a650 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.634+0000 7f2e327fc640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 0x7f2e1c07b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.636 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.634+0000 7f2e327fc640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 0x7f2e1c07b040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e24034000 tx=0x7f2e24004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.638+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2e2c021020 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.818 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.816+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f2e3c0cf7f0 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.820+0000 7f2e38ff9640 1 -- 172.21.15.105:0/2579799503 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f2e2c062b10 con 0x7f2e3c0cf1b0 2024-09-21T07:27:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 msgr2=0x7f2e1c07b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 0x7f2e1c07b040 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2e24034000 tx=0x7f2e24004730 comp rx=0 tx=0).stop 2024-09-21T07:27:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 msgr2=0x7f2e3c15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00c400 tx=0x7f2e2c00c800 comp rx=0 tx=0).stop 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 shutdown_connections 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e3c0d4460 0x7f2e3c1f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2e1c078b30 0x7f2e1c07b040 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2e3c0d3020 0x7f2e3c1f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 --2- 172.21.15.105:0/2579799503 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2e3c0cf1b0 0x7f2e3c15e260 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 >> 172.21.15.105:0/2579799503 conn(0x7f2e3c08bfb0 msgr2=0x7f2e3c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 shutdown_connections 2024-09-21T07:27:32.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.822+0000 7f2e418c5640 1 -- 172.21.15.105:0/2579799503 wait complete. 2024-09-21T07:27:32.835 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-09-21T07:27:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 -- 172.21.15.105:0/3944610777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c400b6850 msgr2=0x7f6c400cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/3944610777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c400b6850 0x7f6c400cf760 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c400ba400 tx=0x7f6c3802a800 comp rx=0 tx=0).stop 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 -- 172.21.15.105:0/3944610777 shutdown_connections 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/3944610777 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c400b6850 0x7f6c400cf760 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 -- 172.21.15.105:0/3944610777 >> 172.21.15.105:0/3944610777 conn(0x7f6c4008bfb0 msgr2=0x7f6c400d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 -- 172.21.15.105:0/3944610777 shutdown_connections 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.952+0000 7f6c47fc6640 1 -- 172.21.15.105:0/3944610777 wait complete. 2024-09-21T07:27:32.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.954+0000 7f6c47fc6640 1 Processor -- start 2024-09-21T07:27:32.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.954+0000 7f6c47fc6640 1 -- start start 2024-09-21T07:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c400b6440 0x7f6c400b7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c400b6850 0x7f6c40156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40169f30 con 0x7f6c4008c3c0 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400b3b60 con 0x7f6c400b6850 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c47fc6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4016a0b0 con 0x7f6c400b6440 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.955+0000 7f6c45d62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53520/0 (socket says 172.21.15.105:53520) 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 -- 172.21.15.105:0/1765810099 learned_addr learned my addr 172.21.15.105:0/1765810099 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c46563640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c400b6850 0x7f6c40156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45561640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c400b6440 0x7f6c400b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 -- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c400b6440 msgr2=0x7f6c400b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c400b6440 0x7f6c400b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 -- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c400b6850 msgr2=0x7f6c40156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c400b6850 0x7f6c40156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 -- 172.21.15.105:0/1765810099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c401620f0 con 0x7f6c4008c3c0 2024-09-21T07:27:32.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c45d62640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c3c0 tx=0x7f6c3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c017070 con 0x7f6c4008c3c0 2024-09-21T07:27:32.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c3c0035f0 con 0x7f6c4008c3c0 2024-09-21T07:27:32.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c400b7a90 con 0x7f6c4008c3c0 2024-09-21T07:27:32.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c003a40 con 0x7f6c4008c3c0 2024-09-21T07:27:32.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.956+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c401609c0 con 0x7f6c4008c3c0 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.957+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4006f7f0 con 0x7f6c4008c3c0 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.958+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) ==== 101385+0+0 (secure 0 0 0) 0x7f6c3c01a070 con 0x7f6c4008c3c0 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.958+0000 7f6c367fc640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 0x7f6c2407b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.959+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f6c3c099e70 con 0x7f6c4008c3c0 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.959+0000 7f6c45561640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 0x7f6c2407b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:32.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.959+0000 7f6c45561640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 0x7f6c2407b040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c40157eb0 tx=0x7f6c30009060 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:32.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:32.963+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c401620f0 con 0x7f6c4008c3c0 2024-09-21T07:27:33.213 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:33.211+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f6c400b6d80 con 0x7f6c4008c3c0 2024-09-21T07:27:34.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.007+0000 7f6c367fc640 1 -- 172.21.15.105:0/1765810099 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v92) ==== 155+0+0 (secure 0 0 0) 0x7f6c400cfca0 con 0x7f6c4008c3c0 2024-09-21T07:27:34.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.009+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 msgr2=0x7f6c2407b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.009+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 0x7f6c2407b040 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c40157eb0 tx=0x7f6c30009060 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.009+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 msgr2=0x7f6c40152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.009+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00c3c0 tx=0x7f6c3c00c880 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 shutdown_connections 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c400b6850 0x7f6c40156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6c24078b30 0x7f6c2407b040 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6c400b6440 0x7f6c400b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 --2- 172.21.15.105:0/1765810099 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6c4008c3c0 0x7f6c40152c10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:34.012 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.010+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 >> 172.21.15.105:0/1765810099 conn(0x7f6c4008bfb0 msgr2=0x7f6c400b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:34.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.011+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 shutdown_connections 2024-09-21T07:27:34.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:34.011+0000 7f6c47fc6640 1 -- 172.21.15.105:0/1765810099 wait complete. 2024-09-21T07:27:34.026 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2024-09-21T07:27:34.027 INFO:tasks.cephfs_test_runner: 2024-09-21T07:27:34.027 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T07:27:34.027 INFO:tasks.cephfs_test_runner:Ran 1 test in 41.882s 2024-09-21T07:27:34.027 INFO:tasks.cephfs_test_runner: 2024-09-21T07:27:34.027 INFO:tasks.cephfs_test_runner:OK 2024-09-21T07:27:34.027 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T07:27:34.036 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T07:27:34.062 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T07:27:34.062 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:34.062 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:34.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1257 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:27:34.080 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-09-21T07:27:34.080 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:34.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056024,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1257 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248728k,nr_inodes=812182,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497460k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:27:34.137 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:27:34.137 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:34.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15681 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:27:34.154 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-09-21T07:27:34.154 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:27:34.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056009,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15681 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:27:34.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.211 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:27:34.226 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:27:34.227 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:27:34.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.227 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:27:34.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.293 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:27:34.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.358 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-21T07:27:34.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.464 DEBUG:teuthology.orchestra.run.smithi105:> ip netns list 2024-09-21T07:27:34.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo ip link delete ceph-brx 2024-09-21T07:27:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:Cannot find device "ceph-brx" 2024-09-21T07:27:34.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:27:34.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.549 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:27:34.565 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:27:34.566 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:27:34.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:27:34.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.632 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:27:34.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T07:27:34.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.803 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-09-21T07:27:34.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:27:34.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-09-21T07:27:34.885 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-09-21T07:27:34.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:27:34.887 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T07:27:34.896 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:27:34.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 -- 172.21.15.105:0/3212287533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 msgr2=0x7f55000ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 --2- 172.21.15.105:0/3212287533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f55000ca760 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54f0005fd0 tx=0x7f54f002a540 comp rx=0 tx=0).stop 2024-09-21T07:27:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 -- 172.21.15.105:0/3212287533 shutdown_connections 2024-09-21T07:27:35.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 --2- 172.21.15.105:0/3212287533 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550006f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 --2- 172.21.15.105:0/3212287533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f55000ca760 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 -- 172.21.15.105:0/3212287533 >> 172.21.15.105:0/3212287533 conn(0x7f55000ccb90 msgr2=0x7f55000ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 -- 172.21.15.105:0/3212287533 shutdown_connections 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.012+0000 7f5505c69640 1 -- 172.21.15.105:0/3212287533 wait complete. 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.014+0000 7f5505c69640 1 Processor -- start 2024-09-21T07:27:35.015 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.014+0000 7f5505c69640 1 -- start start 2024-09-21T07:27:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f550014a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f550015f030 con 0x7f55000cba00 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55000700f0 con 0x7f5500074d00 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f5505c69640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f550015f1b0 con 0x7f55000cb5f0 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f54fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f54fffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53526/0 (socket says 172.21.15.105:53526) 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f54fffff640 1 -- 172.21.15.105:0/2241747204 learned_addr learned my addr 172.21.15.105:0/2241747204 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.015+0000 7f54ff7fe640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f550014a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54feffd640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 -- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 msgr2=0x7f550014b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550014b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 -- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 msgr2=0x7f550014a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f550014a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 -- 172.21.15.105:0/2241747204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550014a590 con 0x7f55000cba00 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54ff7fe640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f550014a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54feffd640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550014b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54fffff640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c730 tx=0x7f54f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8003b90 con 0x7f55000cba00 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54f8003d50 con 0x7f55000cba00 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550009cea0 con 0x7f55000cba00 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f8005330 con 0x7f55000cba00 2024-09-21T07:27:35.020 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.016+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550009d460 con 0x7f55000cba00 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.017+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550009cc10 con 0x7f55000cba00 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.018+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f54f80056e0 con 0x7f55000cba00 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.018+0000 7f54e7fff640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 0x7f54ec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.019+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f54f809ada0 con 0x7f55000cba00 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.019+0000 7f54ff7fe640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 0x7f54ec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.021 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.019+0000 7f54ff7fe640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 0x7f54ec07b0d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f0033000 tx=0x7f54f0004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.025 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.023+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54f80630f0 con 0x7f55000cba00 2024-09-21T07:27:35.199 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.197+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55000711c0 con 0x7f54ec078bc0 2024-09-21T07:27:35.200 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.198+0000 7f54e7fff640 1 -- 172.21.15.105:0/2241747204 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7f55000711c0 con 0x7f54ec078bc0 2024-09-21T07:27:35.200 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:35.200 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:35.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 msgr2=0x7f54ec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 0x7f54ec07b0d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f0033000 tx=0x7f54f0004730 comp rx=0 tx=0).stop 2024-09-21T07:27:35.202 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 msgr2=0x7f550014b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54f800c730 tx=0x7f54f800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 shutdown_connections 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54ec078bc0 0x7f54ec07b0d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55000cba00 0x7f550014b5f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55000cb5f0 0x7f550014b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 --2- 172.21.15.105:0/2241747204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5500074d00 0x7f550014a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 >> 172.21.15.105:0/2241747204 conn(0x7f55000ccb90 msgr2=0x7f55000aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 shutdown_connections 2024-09-21T07:27:35.203 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.200+0000 7f5505c69640 1 -- 172.21.15.105:0/2241747204 wait complete. 2024-09-21T07:27:35.213 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-09-21T07:27:33.488849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226944,"kb_used_data":6552,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747776,"statfs":{"total":773094113280,"available":772861722624,"internally_reserved":0,"allocated":6709248,"data_stored":4413620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732314,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-356,"num_write_kb":-2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991753"},"pg_stats":[{"pgid":"1.0","version":"42'79","reported_seq":178,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T07:27:29.480888+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:29.480888+0000","last_peered":"2024-09-21T07:27:29.480888+0000","last_clean":"2024-09-21T07:27:29.480888+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:29.480888+0000","last_undegraded":"2024-09-21T07:27:29.480888+0000","last_fullsized":"2024-09-21T07:27:29.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1454080,"data_stored":1153185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":225130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":617876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":225253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":234650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1232896,"data_stored":946066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":210574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":800886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:35.214 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 -- 172.21.15.105:0/3085132770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6850 msgr2=0x7f19b80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/3085132770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6850 0x7f19b80cf590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f19b0005fd0 tx=0x7f19b002a7a0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 -- 172.21.15.105:0/3085132770 shutdown_connections 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/3085132770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6850 0x7f19b80cf590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/3085132770 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b80b6440 0x7f19b80cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 -- 172.21.15.105:0/3085132770 >> 172.21.15.105:0/3085132770 conn(0x7f19b808bfb0 msgr2=0x7f19b80d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 -- 172.21.15.105:0/3085132770 shutdown_connections 2024-09-21T07:27:35.326 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.323+0000 7f19bd2c4640 1 -- 172.21.15.105:0/3085132770 wait complete. 2024-09-21T07:27:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.325+0000 7f19bd2c4640 1 Processor -- start 2024-09-21T07:27:35.327 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.325+0000 7f19bd2c4640 1 -- start start 2024-09-21T07:27:35.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b808c3c0 0x7f19b8150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6440 0x7f19b8150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80cfc50 con 0x7f19b80b6850 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80cfad0 con 0x7f19b80b6440 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.326+0000 7f19bd2c4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f19b8162c70 con 0x7f19b808c3c0 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53538/0 (socket says 172.21.15.105:53538) 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 -- 172.21.15.105:0/1337971388 learned_addr learned my addr 172.21.15.105:0/1337971388 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b6ffd640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b808c3c0 0x7f19b8150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b67fc640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6440 0x7f19b8150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 -- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b808c3c0 msgr2=0x7f19b8150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b808c3c0 0x7f19b8150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 -- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6440 msgr2=0x7f19b8150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6440 0x7f19b8150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 -- 172.21.15.105:0/1337971388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b80cbad0 con 0x7f19b80b6850 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19b77fe640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009cd0 tx=0x7f19b0032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0041020 con 0x7f19b80b6850 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b0004040 con 0x7f19b80b6850 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b80cdc30 con 0x7f19b80b6850 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b0036040 con 0x7f19b80b6850 2024-09-21T07:27:35.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.327+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b80ce1f0 con 0x7f19b80b6850 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.328+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b806f7f0 con 0x7f19b80b6850 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.329+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f19b0044020 con 0x7f19b80b6850 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.330+0000 7f199f7fe640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 0x7f19a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.330+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f19b00bbbc0 con 0x7f19b80b6850 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.330+0000 7f19b6ffd640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 0x7f19a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.331+0000 7f19b6ffd640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 0x7f19a407b0a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19a8001eb0 tx=0x7f19a8009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.334+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19b004b070 con 0x7f19b80b6850 2024-09-21T07:27:35.515 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.513+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19b80cb860 con 0x7f19a4078b90 2024-09-21T07:27:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.514+0000 7f199f7fe640 1 -- 172.21.15.105:0/1337971388 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7f19b80cb860 con 0x7f19a4078b90 2024-09-21T07:27:35.516 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:35.516 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 msgr2=0x7f19a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.518 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 0x7f19a407b0a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f19a8001eb0 tx=0x7f19a8009040 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 msgr2=0x7f19b81f4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009cd0 tx=0x7f19b0032be0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 shutdown_connections 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19b80b6440 0x7f19b8150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f19a4078b90 0x7f19a407b0a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f19b80b6850 0x7f19b81f4f90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 --2- 172.21.15.105:0/1337971388 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19b808c3c0 0x7f19b8150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 >> 172.21.15.105:0/1337971388 conn(0x7f19b808bfb0 msgr2=0x7f19b80b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 shutdown_connections 2024-09-21T07:27:35.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.516+0000 7f19bd2c4640 1 -- 172.21.15.105:0/1337971388 wait complete. 2024-09-21T07:27:35.529 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-21T07:27:35.489257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226944,"kb_used_data":6552,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747776,"statfs":{"total":773094113280,"available":772861722624,"internally_reserved":0,"allocated":6709248,"data_stored":4413620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732314,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-356,"num_write_kb":-2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992160"},"pg_stats":[{"pgid":"1.0","version":"42'79","reported_seq":178,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T07:27:29.480888+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:29.480888+0000","last_peered":"2024-09-21T07:27:29.480888+0000","last_clean":"2024-09-21T07:27:29.480888+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:29.480888+0000","last_undegraded":"2024-09-21T07:27:29.480888+0000","last_fullsized":"2024-09-21T07:27:29.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1454080,"data_stored":1153185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":225130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":617876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":225253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":234650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1232896,"data_stored":946066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":210574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":800886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:35.530 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:27:35.530 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 -- 172.21.15.105:0/2412750466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0cc1d0 msgr2=0x7f86dc0cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 --2- 172.21.15.105:0/2412750466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0cc1d0 0x7f86dc0cdf20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0729d0 tx=0x7f86d402aa30 comp rx=0 tx=0).stop 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 -- 172.21.15.105:0/2412750466 shutdown_connections 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 --2- 172.21.15.105:0/2412750466 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0cc1d0 0x7f86dc0cdf20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 --2- 172.21.15.105:0/2412750466 >> v2:172.21.15.112:3300/0 conn(0x7f86dc0cb4a0 0x7f86dc073dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 -- 172.21.15.105:0/2412750466 >> 172.21.15.105:0/2412750466 conn(0x7f86dc0ccb90 msgr2=0x7f86dc0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 -- 172.21.15.105:0/2412750466 shutdown_connections 2024-09-21T07:27:35.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.641+0000 7f86e232b640 1 -- 172.21.15.105:0/2412750466 wait complete. 2024-09-21T07:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.643+0000 7f86e232b640 1 Processor -- start 2024-09-21T07:27:35.644 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.643+0000 7f86e232b640 1 -- start start 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86dc0cb4a0 0x7f86dc09ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86dc1f68b0 0x7f86dc09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0cea80 con 0x7f86dc0ca760 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f86dc0ce900 con 0x7f86dc1f68b0 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e232b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f86dc0cec00 con 0x7f86dc0cb4a0 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53568/0 (socket says 172.21.15.105:53568) 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 -- 172.21.15.105:0/2727754025 learned_addr learned my addr 172.21.15.105:0/2727754025 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86e08c8640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86dc1f68b0 0x7f86dc09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86d3fff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86dc0cb4a0 0x7f86dc09ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 -- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86dc0cb4a0 msgr2=0x7f86dc09ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86dc0cb4a0 0x7f86dc09ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 -- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86dc1f68b0 msgr2=0x7f86dc09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86dc1f68b0 0x7f86dc09ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.644+0000 7f86dbfff640 1 -- 172.21.15.105:0/2727754025 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86dc1f32c0 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86dbfff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00e7c0 tx=0x7f86cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc00ccc0 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86cc00ce80 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86dc1f7090 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86cc005280 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.645+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86dc1f7650 con 0x7f86dc0ca760 2024-09-21T07:27:35.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.646+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86dc0cf120 con 0x7f86dc0ca760 2024-09-21T07:27:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.647+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f86cc005420 con 0x7f86dc0ca760 2024-09-21T07:27:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.647+0000 7f86d97fa640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 0x7f86bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.647+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f86cc09b5c0 con 0x7f86dc0ca760 2024-09-21T07:27:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.647+0000 7f86d3fff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 0x7f86bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.648+0000 7f86d3fff640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 0x7f86bc07b0a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86dc15d200 tx=0x7f86c4009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.651+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86dc1f32c0 con 0x7f86dc0ca760 2024-09-21T07:27:35.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.825+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86dc076d70 con 0x7f86bc078b90 2024-09-21T07:27:35.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.826+0000 7f86d97fa640 1 -- 172.21.15.105:0/2727754025 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13431 (secure 0 0 0) 0x7f86dc076d70 con 0x7f86bc078b90 2024-09-21T07:27:35.828 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:35.828 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:35.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 msgr2=0x7f86bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 0x7f86bc07b0a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86dc15d200 tx=0x7f86c4009040 comp rx=0 tx=0).stop 2024-09-21T07:27:35.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 msgr2=0x7f86dc0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00e7c0 tx=0x7f86cc00ec80 comp rx=0 tx=0).stop 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 shutdown_connections 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86dc1f68b0 0x7f86dc09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f86bc078b90 0x7f86bc07b0a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86dc0cb4a0 0x7f86dc09ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 --2- 172.21.15.105:0/2727754025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86dc0ca760 0x7f86dc0b5a20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 >> 172.21.15.105:0/2727754025 conn(0x7f86dc0ccb90 msgr2=0x7f86dc0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 shutdown_connections 2024-09-21T07:27:35.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.828+0000 7f86e232b640 1 -- 172.21.15.105:0/2727754025 wait complete. 2024-09-21T07:27:35.841 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-09-21T07:27:35.489257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":226944,"kb_used_data":6552,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754747776,"statfs":{"total":773094113280,"available":772861722624,"internally_reserved":0,"allocated":6709248,"data_stored":4413620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732314,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-417,"num_read_kb":-371,"num_write":-356,"num_write_kb":-2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992160"},"pg_stats":[{"pgid":"1.0","version":"42'79","reported_seq":178,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-21T07:27:29.480888+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:29.480888+0000","last_peered":"2024-09-21T07:27:29.480888+0000","last_clean":"2024-09-21T07:27:29.480888+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:29.480888+0000","last_undegraded":"2024-09-21T07:27:29.480888+0000","last_fullsized":"2024-09-21T07:27:29.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1454080,"data_stored":1153185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":507904,"data_stored":225130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":617876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":516096,"data_stored":225253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":234650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1232896,"data_stored":946066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":483328,"data_stored":210574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1077248,"data_stored":800886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:35.842 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 -- 172.21.15.105:0/2550763305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 msgr2=0x7fda64003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 --2- 172.21.15.105:0/2550763305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda64003ec0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009b50 tx=0x7fda64005830 comp rx=0 tx=0).stop 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 -- 172.21.15.105:0/2550763305 shutdown_connections 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 --2- 172.21.15.105:0/2550763305 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda64003ec0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 -- 172.21.15.105:0/2550763305 >> 172.21.15.105:0/2550763305 conn(0x7fda7c0cedb0 msgr2=0x7fda7c0cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 -- 172.21.15.105:0/2550763305 shutdown_connections 2024-09-21T07:27:35.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.953+0000 7fda830dc640 1 -- 172.21.15.105:0/2550763305 wait complete. 2024-09-21T07:27:35.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.954+0000 7fda830dc640 1 Processor -- start 2024-09-21T07:27:35.956 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.954+0000 7fda830dc640 1 -- start start 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda7c15ead0 0x7fda7c1f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda7c1f91f0 0x7fda7c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0cebe0 con 0x7fda64003a50 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0cea60 con 0x7fda7c15ead0 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda830dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0b17d0 con 0x7fda7c1f91f0 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53590/0 (socket says 172.21.15.105:53590) 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 -- 172.21.15.105:0/3497296428 learned_addr learned my addr 172.21.15.105:0/3497296428 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda81679640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda7c1f91f0 0x7fda7c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda7bfff640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda7c15ead0 0x7fda7c1f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 -- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda7c1f91f0 msgr2=0x7fda7c06f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda7c1f91f0 0x7fda7c06f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 -- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda7c15ead0 msgr2=0x7fda7c1f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda7c15ead0 0x7fda7c1f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.956+0000 7fda80e78640 1 -- 172.21.15.105:0/3497296428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c15e570 con 0x7fda64003a50 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda80e78640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda6c036040 tx=0x7fda6c0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c038040 con 0x7fda64003a50 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda6c041070 con 0x7fda64003a50 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c1f9600 con 0x7fda64003a50 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6c03c430 con 0x7fda64003a50 2024-09-21T07:27:35.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.957+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c15e290 con 0x7fda64003a50 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.958+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c0b1c70 con 0x7fda64003a50 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.958+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7fda6c03c5d0 con 0x7fda64003a50 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.959+0000 7fda797fa640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 0x7fda6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.959+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7fda6c0bb780 con 0x7fda64003a50 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.959+0000 7fda7bfff640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 0x7fda6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:35.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.960+0000 7fda7bfff640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 0x7fda6807b0a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda7c073e10 tx=0x7fda70006ef0 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:35.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:35.963+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda7c1f9600 con 0x7fda64003a50 2024-09-21T07:27:36.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.143+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda7c0b3c00 con 0x7fda64003a50 2024-09-21T07:27:36.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.143+0000 7fda797fa640 1 -- 172.21.15.105:0/3497296428 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+14272 (secure 0 0 0) 0x7fda6c0835a0 con 0x7fda64003a50 2024-09-21T07:27:36.145 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:36.146 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":46,"fsid":"15ee7004-a3e2-41ce-b1eb-6358972a1ae2","created":"2024-09-21T07:25:50.395878+0000","modified":"2024-09-21T07:27:32.490344+0000","last_up_change":"2024-09-21T07:26:03.882963+0000","last_in_change":"2024-09-21T07:25:53.955886+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:26:04.782684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"066840df-9c4d-4c30-9039-9c24adc15c08","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6825","nonce":2340388403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6827","nonce":2340388403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6831","nonce":2340388403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":2340388403},{"type":"v1","addr":"172.21.15.105:6829","nonce":2340388403}]},"public_addr":"172.21.15.105:6825/2340388403","cluster_addr":"172.21.15.105:6827/2340388403","heartbeat_back_addr":"172.21.15.105:6831/2340388403","heartbeat_front_addr":"172.21.15.105:6829/2340388403","state":["exists","up"]},{"osd":1,"uuid":"e1787ece-29de-4aca-9dce-d142efa23f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6817","nonce":4277636459}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6819","nonce":4277636459}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6823","nonce":4277636459}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":4277636459},{"type":"v1","addr":"172.21.15.105:6821","nonce":4277636459}]},"public_addr":"172.21.15.105:6817/4277636459","cluster_addr":"172.21.15.105:6819/4277636459","heartbeat_back_addr":"172.21.15.105:6823/4277636459","heartbeat_front_addr":"172.21.15.105:6821/4277636459","state":["exists","up"]},{"osd":2,"uuid":"dff6f411-0843-43ba-a7a3-d855a7a96eac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6809","nonce":3103137694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6811","nonce":3103137694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6815","nonce":3103137694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":3103137694},{"type":"v1","addr":"172.21.15.105:6813","nonce":3103137694}]},"public_addr":"172.21.15.105:6809/3103137694","cluster_addr":"172.21.15.105:6811/3103137694","heartbeat_back_addr":"172.21.15.105:6815/3103137694","heartbeat_front_addr":"172.21.15.105:6813/3103137694","state":["exists","up"]},{"osd":3,"uuid":"a9ea30f8-69d0-4b00-9c3a-c179c73733e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6801","nonce":2773911429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6803","nonce":2773911429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6807","nonce":2773911429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":2773911429},{"type":"v1","addr":"172.21.15.105:6805","nonce":2773911429}]},"public_addr":"172.21.15.105:6801/2773911429","cluster_addr":"172.21.15.105:6803/2773911429","heartbeat_back_addr":"172.21.15.105:6807/2773911429","heartbeat_front_addr":"172.21.15.105:6805/2773911429","state":["exists","up"]},{"osd":4,"uuid":"3a5b5afe-ea5d-4d28-b083-763b9805cec2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6811","nonce":4146883683}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6813","nonce":4146883683}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6817","nonce":4146883683}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":4146883683},{"type":"v1","addr":"172.21.15.112:6815","nonce":4146883683}]},"public_addr":"172.21.15.112:6811/4146883683","cluster_addr":"172.21.15.112:6813/4146883683","heartbeat_back_addr":"172.21.15.112:6817/4146883683","heartbeat_front_addr":"172.21.15.112:6815/4146883683","state":["exists","up"]},{"osd":5,"uuid":"8a7765c0-85a7-4dde-805a-88680cc72d23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6827","nonce":2151662407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6829","nonce":2151662407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6833","nonce":2151662407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2151662407},{"type":"v1","addr":"172.21.15.112:6831","nonce":2151662407}]},"public_addr":"172.21.15.112:6827/2151662407","cluster_addr":"172.21.15.112:6829/2151662407","heartbeat_back_addr":"172.21.15.112:6833/2151662407","heartbeat_front_addr":"172.21.15.112:6831/2151662407","state":["exists","up"]},{"osd":6,"uuid":"efa5ac76-72bc-4495-9353-b42e9421b46b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6819","nonce":2596982695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6821","nonce":2596982695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6825","nonce":2596982695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":2596982695},{"type":"v1","addr":"172.21.15.112:6823","nonce":2596982695}]},"public_addr":"172.21.15.112:6819/2596982695","cluster_addr":"172.21.15.112:6821/2596982695","heartbeat_back_addr":"172.21.15.112:6825/2596982695","heartbeat_front_addr":"172.21.15.112:6823/2596982695","state":["exists","up"]},{"osd":7,"uuid":"52bd24cd-a720-4853-97f8-c5ceb4013988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6803","nonce":3904329758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6805","nonce":3904329758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6809","nonce":3904329758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":3904329758},{"type":"v1","addr":"172.21.15.112:6807","nonce":3904329758}]},"public_addr":"172.21.15.112:6803/3904329758","cluster_addr":"172.21.15.112:6805/3904329758","heartbeat_back_addr":"172.21.15.112:6809/3904329758","heartbeat_front_addr":"172.21.15.112:6807/3904329758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.855734+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.343940+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.502741+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.049706+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.495305+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:01.239984+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.346018+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:26:00.274298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6800/1472023028":"2024-09-22T07:27:29.472636+0000","172.21.15.112:6800/1954024229":"2024-09-22T07:27:22.915542+0000","172.21.15.112:6801/1954024229":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/1737949316":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/490505645":"2024-09-22T07:27:29.472636+0000","172.21.15.112:0/1978324892":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/3405859733":"2024-09-22T07:27:29.472636+0000","172.21.15.112:0/3141574349":"2024-09-22T07:27:29.472636+0000","172.21.15.112:0/3749496995":"2024-09-22T07:27:22.915542+0000","172.21.15.105:6833/1752443650":"2024-09-22T07:27:29.187176+0000","172.21.15.112:6801/1472023028":"2024-09-22T07:27:29.472636+0000","172.21.15.112:0/1565393764":"2024-09-22T07:27:29.472636+0000","172.21.15.112:0/2921736159":"2024-09-22T07:27:22.915542+0000","172.21.15.112:0/3029215345":"2024-09-22T07:27:29.472636+0000","172.21.15.105:6832/1752443650":"2024-09-22T07:27:29.187176+0000","172.21.15.112:0/3528078888":"2024-09-22T07:27:29.472636+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 msgr2=0x7fda6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 0x7fda6807b0a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fda7c073e10 tx=0x7fda70006ef0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 msgr2=0x7fda7c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fda6c036040 tx=0x7fda6c0336f0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 shutdown_connections 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda7c15ead0 0x7fda7c1f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fda68078b90 0x7fda6807b0a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda7c1f91f0 0x7fda7c06f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 --2- 172.21.15.105:0/3497296428 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fda64003a50 0x7fda7c15d5d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 >> 172.21.15.105:0/3497296428 conn(0x7fda7c0cedb0 msgr2=0x7fda7c153990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 shutdown_connections 2024-09-21T07:27:36.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:36.146+0000 7fda830dc640 1 -- 172.21.15.105:0/3497296428 wait complete. 2024-09-21T07:27:37.160 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T07:27:37.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 -- 172.21.15.105:0/1088891825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500b7960 msgr2=0x7f1e500b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 --2- 172.21.15.105:0/1088891825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500b7960 0x7f1e500b17d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1e500ccaf0 tx=0x7f1e4c02a560 comp rx=0 tx=0).stop 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 -- 172.21.15.105:0/1088891825 shutdown_connections 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 --2- 172.21.15.105:0/1088891825 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e500cff00 0x7f1e500d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 --2- 172.21.15.105:0/1088891825 >> v2:172.21.15.112:3300/0 conn(0x7f1e500d0a80 0x7f1e500d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 --2- 172.21.15.105:0/1088891825 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500b7960 0x7f1e500b17d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.289+0000 7f1e56fba640 1 -- 172.21.15.105:0/1088891825 >> 172.21.15.105:0/1088891825 conn(0x7f1e500cedb0 msgr2=0x7f1e500cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.290+0000 7f1e56fba640 1 -- 172.21.15.105:0/1088891825 shutdown_connections 2024-09-21T07:27:37.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.290+0000 7f1e56fba640 1 -- 172.21.15.105:0/1088891825 wait complete. 2024-09-21T07:27:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.291+0000 7f1e56fba640 1 Processor -- start 2024-09-21T07:27:37.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.291+0000 7f1e56fba640 1 -- start start 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e500b7960 0x7f1e5014a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 0x7f1e5015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e500d0a80 0x7f1e5015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500cebe0 con 0x7f1e500cff00 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500cea60 con 0x7f1e500b7960 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e56fba640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1e50160a40 con 0x7f1e500d0a80 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 0x7f1e5015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 0x7f1e5015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53608/0 (socket says 172.21.15.105:53608) 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 -- 172.21.15.105:0/3096228082 learned_addr learned my addr 172.21.15.105:0/3096228082 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e54d56640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e500b7960 0x7f1e5014a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e55557640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e500d0a80 0x7f1e5015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 -- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e500d0a80 msgr2=0x7f1e5015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1e500d0a80 0x7f1e5015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 -- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e500b7960 msgr2=0x7f1e5014a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1e500b7960 0x7f1e5014a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5014ac20 con 0x7f1e500cff00 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.293+0000 7f1e47fff640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 0x7f1e5015f6a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000d6a0 tx=0x7f1e4000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.294+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40017070 con 0x7f1e500cff00 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.294+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e40004310 con 0x7f1e500cff00 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.294+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5014af00 con 0x7f1e500cff00 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.294+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40005280 con 0x7f1e500cff00 2024-09-21T07:27:37.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.294+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5014b4c0 con 0x7f1e500cff00 2024-09-21T07:27:37.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.295+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e500d0310 con 0x7f1e500cff00 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f1e40005420 con 0x7f1e500cff00 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1e3c078b90 0x7f1e3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f1e4009b750 con 0x7f1e500cff00 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e54d56640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1e3c078b90 0x7f1e3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f1e3c07f5d0 0x7f1e3c081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e40016ec0 con 0x7f1e3c07f5d0 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1e4009bbe0 con 0x7f1e500cff00 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.296+0000 7f1e55557640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f1e3c07f5d0 0x7f1e3c081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.297+0000 7f1e54d56640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1e3c078b90 0x7f1e3c07b0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.297+0000 7f1e55557640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f1e3c07f5d0 0x7f1e3c081a60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.297+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== osd.0 v2:172.21.15.105:6824/2340388403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e40016ec0 con 0x7f1e3c07f5d0 2024-09-21T07:27:37.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 --> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1e501f76c0 con 0x7f1e3c07f5d0 2024-09-21T07:27:37.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e457fa640 1 -- 172.21.15.105:0/3096228082 <== osd.0 v2:172.21.15.105:6824/2340388403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1e501f76c0 con 0x7f1e3c07f5d0 2024-09-21T07:27:37.312 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:37.312 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:37.312 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:37.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f1e3c07f5d0 msgr2=0x7f1e3c081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:6824/2340388403,v1:172.21.15.105:6825/2340388403] conn(0x7f1e3c07f5d0 0x7f1e3c081a60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1e3c078b90 msgr2=0x7f1e3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1e3c078b90 0x7f1e3c07b0a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c033000 tx=0x7f1e4c004730 comp rx=0 tx=0).stop 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 msgr2=0x7f1e5015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 --2- 172.21.15.105:0/3096228082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1e500cff00 0x7f1e5015f6a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000d6a0 tx=0x7f1e4000db60 comp rx=0 tx=0).stop 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e55557640 1 -- 172.21.15.105:0/3096228082 reap_dead start 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 shutdown_connections 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.310+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 >> 172.21.15.105:0/3096228082 conn(0x7f1e500cedb0 msgr2=0x7f1e500b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.311+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 shutdown_connections 2024-09-21T07:27:37.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.311+0000 7f1e56fba640 1 -- 172.21.15.105:0/3096228082 wait complete. 2024-09-21T07:27:37.323 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 -- 172.21.15.105:0/3730520576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92406fef0 msgr2=0x7fb924073520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 --2- 172.21.15.105:0/3730520576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92406fef0 0x7fb924073520 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb91c005fd0 tx=0x7fb91c02b8f0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 -- 172.21.15.105:0/3730520576 shutdown_connections 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 --2- 172.21.15.105:0/3730520576 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb92406fef0 0x7fb924073520 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 --2- 172.21.15.105:0/3730520576 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb924074d00 0x7fb9240ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 -- 172.21.15.105:0/3730520576 >> 172.21.15.105:0/3730520576 conn(0x7fb9240ccb90 msgr2=0x7fb9240ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 -- 172.21.15.105:0/3730520576 shutdown_connections 2024-09-21T07:27:37.437 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.434+0000 7fb92c084640 1 -- 172.21.15.105:0/3730520576 wait complete. 2024-09-21T07:27:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.436+0000 7fb92c084640 1 Processor -- start 2024-09-21T07:27:37.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.436+0000 7fb92c084640 1 -- start start 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9240cb5f0 0x7fb9241f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9241f7150 0x7fb9240b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406f2e0 con 0x7fb924074d00 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb92406f160 con 0x7fb9241f7150 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92c084640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fb92406f460 con 0x7fb9240cb5f0 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53618/0 (socket says 172.21.15.105:53618) 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 -- 172.21.15.105:0/3995390046 learned_addr learned my addr 172.21.15.105:0/3995390046 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:37.440 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92961f640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9240cb5f0 0x7fb9241f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb92a621640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9241f7150 0x7fb9240b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 -- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9240cb5f0 msgr2=0x7fb9241f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9240cb5f0 0x7fb9241f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 -- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9241f7150 msgr2=0x7fb9240b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9241f7150 0x7fb9240b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.437+0000 7fb929e20640 1 -- 172.21.15.105:0/3995390046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9241f3140 con 0x7fb924074d00 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb929e20640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb91400e7c0 tx=0x7fb91400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400ccc0 con 0x7fb924074d00 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb91400ce80 con 0x7fb924074d00 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9241f3420 con 0x7fb924074d00 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914005280 con 0x7fb924074d00 2024-09-21T07:27:37.441 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.438+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241f39e0 con 0x7fb924074d00 2024-09-21T07:27:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.439+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb924073520 con 0x7fb924074d00 2024-09-21T07:27:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.439+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7fb914005420 con 0x7fb924074d00 2024-09-21T07:27:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.440+0000 7fb9227fc640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 0x7fb91007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.440+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7fb91409b730 con 0x7fb924074d00 2024-09-21T07:27:37.442 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.440+0000 7fb92961f640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 0x7fb91007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.441+0000 7fb92961f640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 0x7fb91007b0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b5e90 tx=0x7fb918009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.446 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.444+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb914063440 con 0x7fb924074d00 2024-09-21T07:27:37.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.621+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fb924076d70 con 0x7fb910078b90 2024-09-21T07:27:37.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.621+0000 7fb9227fc640 1 -- 172.21.15.105:0/3995390046 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb924076d70 con 0x7fb910078b90 2024-09-21T07:27:37.623 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T07:27:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 msgr2=0x7fb91007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 0x7fb91007b0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b5e90 tx=0x7fb918009040 comp rx=0 tx=0).stop 2024-09-21T07:27:37.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 msgr2=0x7fb9241e1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb91400e7c0 tx=0x7fb91400ec80 comp rx=0 tx=0).stop 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 shutdown_connections 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9241f7150 0x7fb9240b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fb910078b90 0x7fb91007b0a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9240cb5f0 0x7fb9241f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 --2- 172.21.15.105:0/3995390046 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb924074d00 0x7fb9241e1170 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 >> 172.21.15.105:0/3995390046 conn(0x7fb9240ccb90 msgr2=0x7fb92415ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 shutdown_connections 2024-09-21T07:27:37.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.623+0000 7fb92c084640 1 -- 172.21.15.105:0/3995390046 wait complete. 2024-09-21T07:27:37.636 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T07:27:37.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 -- 172.21.15.105:0/1987814519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e406fbf0 msgr2=0x7f54e40b3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 --2- 172.21.15.105:0/1987814519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e406fbf0 0x7f54e40b3420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f54dc005fd0 tx=0x7f54dc02a940 comp rx=0 tx=0).stop 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 -- 172.21.15.105:0/1987814519 shutdown_connections 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 --2- 172.21.15.105:0/1987814519 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e406fbf0 0x7f54e40b3420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 -- 172.21.15.105:0/1987814519 >> 172.21.15.105:0/1987814519 conn(0x7f54e408bfb0 msgr2=0x7f54e40d4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 -- 172.21.15.105:0/1987814519 shutdown_connections 2024-09-21T07:27:37.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.752+0000 7f54e9a1f640 1 -- 172.21.15.105:0/1987814519 wait complete. 2024-09-21T07:27:37.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.753+0000 7f54e9a1f640 1 Processor -- start 2024-09-21T07:27:37.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.754+0000 7f54e9a1f640 1 -- start start 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e406fbf0 0x7f54e40cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 0x7f54e40a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40a16f0 0x7f54e41f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40cfb40 con 0x7f54e40d4e10 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40cf9c0 con 0x7f54e406fbf0 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e9a1f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40cfcc0 con 0x7f54e40a16f0 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 0x7f54e40a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 0x7f54e40a11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53642/0 (socket says 172.21.15.105:53642) 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 -- 172.21.15.105:0/3873465103 learned_addr learned my addr 172.21.15.105:0/3873465103 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e2ffd640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e406fbf0 0x7f54e40cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54e37fe640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40a16f0 0x7f54e41f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 -- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40a16f0 msgr2=0x7f54e41f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f54e40a16f0 0x7f54e41f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 -- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e406fbf0 msgr2=0x7f54e40cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f54e406fbf0 0x7f54e40cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40b9550 con 0x7f54e40d4e10 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.755+0000 7f54da7fc640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 0x7f54e40a11b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c3d0 tx=0x7f54cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc017070 con 0x7f54e40d4e10 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54cc0035f0 con 0x7f54e40d4e10 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e40b9a50 con 0x7f54e40d4e10 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc003a40 con 0x7f54e40d4e10 2024-09-21T07:27:37.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e40a06d0 con 0x7f54e40d4e10 2024-09-21T07:27:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.756+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54e40ce500 con 0x7f54e40d4e10 2024-09-21T07:27:37.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.757+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f54cc01a070 con 0x7f54e40d4e10 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e0ff9640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54c4078bc0 0x7f54c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f54cc09a710 con 0x7f54e40d4e10 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e2ffd640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54c4078bc0 0x7f54c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e0ff9640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f54c407f600 0x7f54c4081a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54cc009e50 con 0x7f54c407f600 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f54cc09aba0 con 0x7f54e40d4e10 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.758+0000 7f54e37fe640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f54c407f600 0x7f54c4081a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.759+0000 7f54e2ffd640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54c4078bc0 0x7f54c407b0d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f54d4000f60 tx=0x7f54d4009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.759+0000 7f54e37fe640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f54c407f600 0x7f54c4081a90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.759+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== osd.1 v2:172.21.15.105:6816/4277636459 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f54cc009e50 con 0x7f54c407f600 2024-09-21T07:27:37.773 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 --> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f54e40b9800 con 0x7f54c407f600 2024-09-21T07:27:37.774 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e0ff9640 1 -- 172.21.15.105:0/3873465103 <== osd.1 v2:172.21.15.105:6816/4277636459 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f54e40b9800 con 0x7f54c407f600 2024-09-21T07:27:37.774 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:37.774 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:37.774 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f54c407f600 msgr2=0x7f54c4081a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:6816/4277636459,v1:172.21.15.105:6817/4277636459] conn(0x7f54c407f600 0x7f54c4081a90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54c4078bc0 msgr2=0x7f54c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.775 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f54c4078bc0 0x7f54c407b0d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f54d4000f60 tx=0x7f54d4009040 comp rx=0 tx=0).stop 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 msgr2=0x7f54e40a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 --2- 172.21.15.105:0/3873465103 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f54e40d4e10 0x7f54e40a11b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c3d0 tx=0x7f54cc00c890 comp rx=0 tx=0).stop 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e37fe640 1 -- 172.21.15.105:0/3873465103 reap_dead start 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 shutdown_connections 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 >> 172.21.15.105:0/3873465103 conn(0x7f54e408bfb0 msgr2=0x7f54e4161af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 shutdown_connections 2024-09-21T07:27:37.776 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.772+0000 7f54e9a1f640 1 -- 172.21.15.105:0/3873465103 wait complete. 2024-09-21T07:27:37.784 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T07:27:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 -- 172.21.15.105:0/2590905838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b36d0 msgr2=0x7f5eec0afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/2590905838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b36d0 0x7f5eec0afa10 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0cd5f0 tx=0x7f5ee802a560 comp rx=0 tx=0).stop 2024-09-21T07:27:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 -- 172.21.15.105:0/2590905838 shutdown_connections 2024-09-21T07:27:37.898 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/2590905838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0be0 0x7f5eec15f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/2590905838 >> v2:172.21.15.112:3300/0 conn(0x7f5eec0b0290 0x7f5eec0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/2590905838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b36d0 0x7f5eec0afa10 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 -- 172.21.15.105:0/2590905838 >> 172.21.15.105:0/2590905838 conn(0x7f5eec09aa20 msgr2=0x7f5eec0b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 -- 172.21.15.105:0/2590905838 shutdown_connections 2024-09-21T07:27:37.899 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.896+0000 7f5ef3b13640 1 -- 172.21.15.105:0/2590905838 wait complete. 2024-09-21T07:27:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.898+0000 7f5ef3b13640 1 Processor -- start 2024-09-21T07:27:37.900 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.898+0000 7f5ef3b13640 1 -- start start 2024-09-21T07:27:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0290 0x7f5eec154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0b36d0 0x7f5eec14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec15f8a0 con 0x7f5eec0b0be0 2024-09-21T07:27:37.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5eec15f720 con 0x7f5eec0b36d0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef3b13640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5eec15fa20 con 0x7f5eec0b0290 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef10ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef10ae640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53654/0 (socket says 172.21.15.105:53654) 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef10ae640 1 -- 172.21.15.105:0/975258201 learned_addr learned my addr 172.21.15.105:0/975258201 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef20b0640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0b36d0 0x7f5eec14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.899+0000 7f5ef18af640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0290 0x7f5eec154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 -- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0290 msgr2=0x7f5eec154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0290 0x7f5eec154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 -- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0b36d0 msgr2=0x7f5eec14dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0b36d0 0x7f5eec14dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 -- 172.21.15.105:0/975258201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eec14e160 con 0x7f5eec0b0be0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef10ae640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00e710 tx=0x7f5edc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00cab0 con 0x7f5eec0b0be0 2024-09-21T07:27:37.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5edc00cd40 con 0x7f5eec0b0be0 2024-09-21T07:27:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc010690 con 0x7f5eec0b0be0 2024-09-21T07:27:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eec15dfb0 con 0x7f5eec0b0be0 2024-09-21T07:27:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.900+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eec15e570 con 0x7f5eec0b0be0 2024-09-21T07:27:37.904 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.901+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eec15dd20 con 0x7f5eec0b0be0 2024-09-21T07:27:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.902+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f5edc010a40 con 0x7f5eec0b0be0 2024-09-21T07:27:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.902+0000 7f5ee27fc640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 0x7f5ed007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.902+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f5edc09b8e0 con 0x7f5eec0b0be0 2024-09-21T07:27:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.903+0000 7f5ef18af640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 0x7f5ed007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:37.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.903+0000 7f5ef18af640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 0x7f5ed007b0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8033000 tx=0x7f5ee8004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:37.908 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:37.907+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5edc063670 con 0x7f5eec0b0be0 2024-09-21T07:27:38.088 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.087+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5eec15ffb0 con 0x7f5ed0078b90 2024-09-21T07:27:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.087+0000 7f5ee27fc640 1 -- 172.21.15.105:0/975258201 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5eec15ffb0 con 0x7f5ed0078b90 2024-09-21T07:27:38.089 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T07:27:38.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 msgr2=0x7f5ed007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 0x7f5ed007b0a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8033000 tx=0x7f5ee8004730 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 msgr2=0x7f5eec09ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00e710 tx=0x7f5edc00ebd0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 shutdown_connections 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eec0b36d0 0x7f5eec14dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5ed0078b90 0x7f5ed007b0a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5eec0b0be0 0x7f5eec09ee90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 --2- 172.21.15.105:0/975258201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5eec0b0290 0x7f5eec154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 >> 172.21.15.105:0/975258201 conn(0x7f5eec09aa20 msgr2=0x7f5eec0d11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 shutdown_connections 2024-09-21T07:27:38.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.089+0000 7f5ef3b13640 1 -- 172.21.15.105:0/975258201 wait complete. 2024-09-21T07:27:38.102 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T07:27:38.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 -- 172.21.15.105:0/845585412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 msgr2=0x7f3f200d2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 --2- 172.21.15.105:0/845585412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f200d2bd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3f2008d470 tx=0x7f3f1802a8f0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 -- 172.21.15.105:0/845585412 shutdown_connections 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 --2- 172.21.15.105:0/845585412 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f200d2bd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 --2- 172.21.15.105:0/845585412 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f200b64f0 0x7f3f200d2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 --2- 172.21.15.105:0/845585412 >> v2:172.21.15.112:3300/0 conn(0x7f3f200b33f0 0x7f3f200b5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 -- 172.21.15.105:0/845585412 >> 172.21.15.105:0/845585412 conn(0x7f3f200d4530 msgr2=0x7f3f200d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 -- 172.21.15.105:0/845585412 shutdown_connections 2024-09-21T07:27:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.216+0000 7f3f27a15640 1 -- 172.21.15.105:0/845585412 wait complete. 2024-09-21T07:27:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.218+0000 7f3f27a15640 1 Processor -- start 2024-09-21T07:27:38.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.218+0000 7f3f27a15640 1 -- start start 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f200b33f0 0x7f3f201e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f200b64f0 0x7f3f2014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f201509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200b9df0 con 0x7f3f200d27c0 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f200b9c70 con 0x7f3f200b64f0 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f27a15640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f3f200d3110 con 0x7f3f200b33f0 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f25fb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f201509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f25fb2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f201509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53668/0 (socket says 172.21.15.105:53668) 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f25fb2640 1 -- 172.21.15.105:0/309340693 learned_addr learned my addr 172.21.15.105:0/309340693 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f257b1640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f200b33f0 0x7f3f201e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f24fb0640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f200b64f0 0x7f3f2014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.219+0000 7f3f25fb2640 1 -- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f200b33f0 msgr2=0x7f3f201e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f25fb2640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3f200b33f0 0x7f3f201e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f25fb2640 1 -- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f200b64f0 msgr2=0x7f3f2014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f25fb2640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f200b64f0 0x7f3f2014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f25fb2640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f200cb8e0 con 0x7f3f200d27c0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f25fb2640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f201509c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f18033040 tx=0x7f3f18003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18004240 con 0x7f3f200d27c0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f200cbbc0 con 0x7f3f200d27c0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f1802d070 con 0x7f3f200d27c0 2024-09-21T07:27:38.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f180046a0 con 0x7f3f200d27c0 2024-09-21T07:27:38.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.220+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2014e5c0 con 0x7f3f200d27c0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.221+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f200d2280 con 0x7f3f200d27c0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f3f18031480 con 0x7f3f200d27c0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f3f04078b90 0x7f3f0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f3f180bc660 con 0x7f3f200d27c0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f257b1640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f3f04078b90 0x7f3f0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7f3f0407f5d0 0x7f3f04081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f1803be90 con 0x7f3f0407f5d0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.222+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3f180c3050 con 0x7f3f200d27c0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.223+0000 7f3f24fb0640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7f3f0407f5d0 0x7f3f04081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.225 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.223+0000 7f3f257b1640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f3f04078b90 0x7f3f0407b0a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005fd0 tx=0x7f3f1c00b260 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.223+0000 7f3f24fb0640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7f3f0407f5d0 0x7f3f04081a60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.226 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.224+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== osd.2 v2:172.21.15.105:6808/3103137694 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f1803be90 con 0x7f3f0407f5d0 2024-09-21T07:27:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.236+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 --> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f200cb670 con 0x7f3f0407f5d0 2024-09-21T07:27:38.238 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.236+0000 7f3f15ffb640 1 -- 172.21.15.105:0/309340693 <== osd.2 v2:172.21.15.105:6808/3103137694 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f200cb670 con 0x7f3f0407f5d0 2024-09-21T07:27:38.238 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:38.238 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:38.239 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.236+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7f3f0407f5d0 msgr2=0x7f3f04081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.236+0000 7f3f27a15640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:6808/3103137694,v1:172.21.15.105:6809/3103137694] conn(0x7f3f0407f5d0 0x7f3f04081a60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f3f04078b90 msgr2=0x7f3f0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f3f04078b90 0x7f3f0407b0a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005fd0 tx=0x7f3f1c00b260 comp rx=0 tx=0).stop 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 msgr2=0x7f3f201509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 --2- 172.21.15.105:0/309340693 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f200d27c0 0x7f3f201509c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f18033040 tx=0x7f3f18003c20 comp rx=0 tx=0).stop 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f25fb2640 1 -- 172.21.15.105:0/309340693 reap_dead start 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 shutdown_connections 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 >> 172.21.15.105:0/309340693 conn(0x7f3f200d4530 msgr2=0x7f3f20161a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 shutdown_connections 2024-09-21T07:27:38.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.237+0000 7f3f27a15640 1 -- 172.21.15.105:0/309340693 wait complete. 2024-09-21T07:27:38.249 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T07:27:38.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 -- 172.21.15.105:0/2937594025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00c5a50 msgr2=0x7febf00d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 --2- 172.21.15.105:0/2937594025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00c5a50 0x7febf00d0170 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7febe00098e0 tx=0x7febe002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 -- 172.21.15.105:0/2937594025 shutdown_connections 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 --2- 172.21.15.105:0/2937594025 >> v2:172.21.15.112:3300/0 conn(0x7febf00ce300 0x7febf00ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 --2- 172.21.15.105:0/2937594025 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00c5a50 0x7febf00d0170 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 -- 172.21.15.105:0/2937594025 >> 172.21.15.105:0/2937594025 conn(0x7febf00cee40 msgr2=0x7febf00cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 -- 172.21.15.105:0/2937594025 shutdown_connections 2024-09-21T07:27:38.363 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.360+0000 7febf7040640 1 -- 172.21.15.105:0/2937594025 wait complete. 2024-09-21T07:27:38.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.362+0000 7febf7040640 1 Processor -- start 2024-09-21T07:27:38.364 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.362+0000 7febf7040640 1 -- start start 2024-09-21T07:27:38.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febf00c5a50 0x7febf00c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.366 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 0x7febf0142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7febf00ccfb0 con 0x7febf00ce300 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febf009bc40 con 0x7febf00d06b0 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.363+0000 7febf7040640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7febf00cd130 con 0x7febf00c5a50 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53678/0 (socket says 172.21.15.105:53678) 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 -- 172.21.15.105:0/1012567593 learned_addr learned my addr 172.21.15.105:0/1012567593 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febf4ddc640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febf00c5a50 0x7febf00c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febf55dd640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 0x7febf0142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 -- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febf00c5a50 msgr2=0x7febf00c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febf00c5a50 0x7febf00c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 -- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 msgr2=0x7febf0142990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 0x7febf0142990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 -- 172.21.15.105:0/1012567593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf01e7c40 con 0x7febf00ce300 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febf55dd640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 0x7febf0142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febeffff640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7febe400c370 tx=0x7febe400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4017070 con 0x7febf00ce300 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febe40053c0 con 0x7febf00ce300 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf01e7f00 con 0x7febf00ce300 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4003850 con 0x7febf00ce300 2024-09-21T07:27:38.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.364+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf0151c60 con 0x7febf00ce300 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.365+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf00d0170 con 0x7febf00ce300 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.366+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7febe401a070 con 0x7febf00ce300 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.366+0000 7febed7fa640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 0x7febdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.367+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7febe4099f90 con 0x7febf00ce300 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.367+0000 7febf4ddc640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 0x7febdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.367+0000 7febf4ddc640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 0x7febdc07b0a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7febe0034000 tx=0x7febe0004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.371+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7febe4061db0 con 0x7febf00ce300 2024-09-21T07:27:38.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.548+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7febf01e8550 con 0x7febdc078b90 2024-09-21T07:27:38.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.549+0000 7febed7fa640 1 -- 172.21.15.105:0/1012567593 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7febf01e8550 con 0x7febdc078b90 2024-09-21T07:27:38.551 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 msgr2=0x7febdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 0x7febdc07b0a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7febe0034000 tx=0x7febe0004730 comp rx=0 tx=0).stop 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 msgr2=0x7febf0142450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7febe400c370 tx=0x7febe400c830 comp rx=0 tx=0).stop 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 shutdown_connections 2024-09-21T07:27:38.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febf00d06b0 0x7febf0142990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7febdc078b90 0x7febdc07b0a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febf00ce300 0x7febf0142450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 --2- 172.21.15.105:0/1012567593 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febf00c5a50 0x7febf00c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 >> 172.21.15.105:0/1012567593 conn(0x7febf00cee40 msgr2=0x7febf0153990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 shutdown_connections 2024-09-21T07:27:38.554 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.551+0000 7febf7040640 1 -- 172.21.15.105:0/1012567593 wait complete. 2024-09-21T07:27:38.564 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T07:27:38.564 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 -- 172.21.15.105:0/190173200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be0165e20 msgr2=0x7f8be0168240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 --2- 172.21.15.105:0/190173200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be0165e20 0x7f8be0168240 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009be0 tx=0x7f8bd802a890 comp rx=0 tx=0).stop 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 -- 172.21.15.105:0/190173200 shutdown_connections 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 --2- 172.21.15.105:0/190173200 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be0165e20 0x7f8be0168240 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 -- 172.21.15.105:0/190173200 >> 172.21.15.105:0/190173200 conn(0x7f8be00cede0 msgr2=0x7f8be00cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 -- 172.21.15.105:0/190173200 shutdown_connections 2024-09-21T07:27:38.680 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.678+0000 7f8be5225640 1 -- 172.21.15.105:0/190173200 wait complete. 2024-09-21T07:27:38.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.680+0000 7f8be5225640 1 Processor -- start 2024-09-21T07:27:38.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.680+0000 7f8be5225640 1 -- start start 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8be00b7990 0x7f8be014f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be00d0360 0x7f8be0164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 0x7f8be01653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0075330 con 0x7f8be0165e20 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00d1310 con 0x7f8be00b7990 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8be5225640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00d3180 con 0x7f8be00d0360 2024-09-21T07:27:38.684 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8bdf59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 0x7f8be01653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8bdf59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 0x7f8be01653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53688/0 (socket says 172.21.15.105:53688) 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.681+0000 7f8bdf59e640 1 -- 172.21.15.105:0/2654271120 learned_addr learned my addr 172.21.15.105:0/2654271120 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bd659c640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be00d0360 0x7f8be0164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bded9d640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8be00b7990 0x7f8be014f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 -- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be00d0360 msgr2=0x7f8be0164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8be00d0360 0x7f8be0164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 -- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8be00b7990 msgr2=0x7f8be014f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8be00b7990 0x7f8be014f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be0150020 con 0x7f8be0165e20 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bded9d640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8be00b7990 0x7f8be014f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bdf59e640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 0x7f8be01653e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009d00 tx=0x7f8bd8033d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8031d80 con 0x7f8be0165e20 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd8003970 con 0x7f8be0165e20 2024-09-21T07:27:38.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01f99a0 con 0x7f8be0165e20 2024-09-21T07:27:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8003d60 con 0x7f8be0165e20 2024-09-21T07:27:38.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.682+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be015c080 con 0x7f8be0165e20 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.683+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8be0156d80 con 0x7f8be0165e20 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.684+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f8bd80330d0 con 0x7f8be0165e20 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.684+0000 7f8bd7fff640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f8bc0078b90 0x7f8bc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f8bd80bc7b0 con 0x7f8be0165e20 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd7fff640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7f8bc007f5d0 0x7f8bc0081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8bd802dea0 con 0x7f8bc007f5d0 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8bd80bcc40 con 0x7f8be0165e20 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bded9d640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f8bc0078b90 0x7f8bc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd659c640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7f8bc007f5d0 0x7f8bc0081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bd659c640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7f8bc007f5d0 0x7f8bc0081a60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.685+0000 7f8bded9d640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f8bc0078b90 0x7f8bc007b0a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8be0073f30 tx=0x7f8bd0009040 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.686+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== osd.3 v2:172.21.15.105:6800/2773911429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8bd802dea0 con 0x7f8bc007f5d0 2024-09-21T07:27:38.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.698+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 --> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8be00b7540 con 0x7f8bc007f5d0 2024-09-21T07:27:38.700 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.698+0000 7f8bd7fff640 1 -- 172.21.15.105:0/2654271120 <== osd.3 v2:172.21.15.105:6800/2773911429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8be00b7540 con 0x7f8bc007f5d0 2024-09-21T07:27:38.700 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:38.700 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:38.700 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7f8bc007f5d0 msgr2=0x7f8bc0081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:6800/2773911429,v1:172.21.15.105:6801/2773911429] conn(0x7f8bc007f5d0 0x7f8bc0081a60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.701 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f8bc0078b90 msgr2=0x7f8bc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f8bc0078b90 0x7f8bc007b0a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8be0073f30 tx=0x7f8bd0009040 comp rx=0 tx=0).stop 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 msgr2=0x7f8be01653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 --2- 172.21.15.105:0/2654271120 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8be0165e20 0x7f8be01653e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009d00 tx=0x7f8bd8033d00 comp rx=0 tx=0).stop 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8bdf59e640 1 -- 172.21.15.105:0/2654271120 reap_dead start 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 shutdown_connections 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 >> 172.21.15.105:0/2654271120 conn(0x7f8be00cede0 msgr2=0x7f8be00b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 shutdown_connections 2024-09-21T07:27:38.702 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.699+0000 7f8be5225640 1 -- 172.21.15.105:0/2654271120 wait complete. 2024-09-21T07:27:38.711 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/933576819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a94004800 msgr2=0x7f6a94004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/933576819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a94004800 0x7f6a94004c10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0001830 tx=0x7f6a94005830 comp rx=0 tx=0).stop 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/933576819 shutdown_connections 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/933576819 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6a94004800 0x7f6a94004c10 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/933576819 >> 172.21.15.105:0/933576819 conn(0x7f6aac09d810 msgr2=0x7f6aac08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/933576819 shutdown_connections 2024-09-21T07:27:38.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.823+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/933576819 wait complete. 2024-09-21T07:27:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.824+0000 7f6ab1a5c640 1 Processor -- start 2024-09-21T07:27:38.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.825+0000 7f6ab1a5c640 1 -- start start 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94004800 0x7f6aac158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6aac0ccf70 0x7f6aac161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b64f0 con 0x7f6aac0ccb60 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6370 con 0x7f6a94004800 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6ab1a5c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac0b6670 con 0x7f6aac0ccf70 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53702/0 (socket says 172.21.15.105:53702) 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 -- 172.21.15.105:0/770310760 learned_addr learned my addr 172.21.15.105:0/770310760 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaaffd640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94004800 0x7f6aac158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aab7fe640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6aac0ccf70 0x7f6aac161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 -- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6aac0ccf70 msgr2=0x7f6aac161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6aac0ccf70 0x7f6aac161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 -- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94004800 msgr2=0x7f6aac158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94004800 0x7f6aac158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 -- 172.21.15.105:0/770310760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac0cb960 con 0x7f6aac0ccb60 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.826+0000 7f6aaa7fc640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0001920 tx=0x7f6aa00060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.827+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0043070 con 0x7f6aac0ccb60 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.827+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6aa0003af0 con 0x7f6aac0ccb60 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.827+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00064c0 con 0x7f6aac0ccb60 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.827+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac0cbc40 con 0x7f6aac0ccb60 2024-09-21T07:27:38.830 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.827+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac0cc200 con 0x7f6aac0ccb60 2024-09-21T07:27:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.828+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac0cb6d0 con 0x7f6aac0ccb60 2024-09-21T07:27:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.829+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f6aa0033480 con 0x7f6aac0ccb60 2024-09-21T07:27:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.829+0000 7f6a93fff640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.829+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f6aa00bac80 con 0x7f6aac0ccb60 2024-09-21T07:27:38.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.829+0000 7f6aaaffd640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:38.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.830+0000 7f6aaaffd640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098b0 tx=0x7f6a9c005f30 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:38.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:38.834+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6aa0082a20 con 0x7f6aac0ccb60 2024-09-21T07:27:39.010 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.008+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f6aac06a170 con 0x7f6a98078b90 2024-09-21T07:27:39.011 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.009+0000 7f6a93fff640 1 -- 172.21.15.105:0/770310760 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6aac06a170 con 0x7f6a98078b90 2024-09-21T07:27:39.011 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 msgr2=0x7f6a9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 0x7f6a9807b0a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c0098b0 tx=0x7f6a9c005f30 comp rx=0 tx=0).stop 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 msgr2=0x7f6aac160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0001920 tx=0x7f6aa00060e0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 shutdown_connections 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6a98078b90 0x7f6a9807b0a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6aac0ccf70 0x7f6aac161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6aac0ccb60 0x7f6aac160fc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 --2- 172.21.15.105:0/770310760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a94004800 0x7f6aac158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 >> 172.21.15.105:0/770310760 conn(0x7f6aac09d810 msgr2=0x7f6aac0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 shutdown_connections 2024-09-21T07:27:39.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.011+0000 7f6ab1a5c640 1 -- 172.21.15.105:0/770310760 wait complete. 2024-09-21T07:27:39.025 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T07:27:39.025 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/2933851497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c34155e00 msgr2=0x7f1c340b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/2933851497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c34155e00 0x7f1c340b34f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009830 tx=0x7f1c2402a510 comp rx=0 tx=0).stop 2024-09-21T07:27:39.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/2933851497 shutdown_connections 2024-09-21T07:27:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/2933851497 >> v2:172.21.15.112:3300/0 conn(0x7f1c340b47f0 0x7f1c340aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/2933851497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c34155e00 0x7f1c340b34f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/2933851497 >> 172.21.15.105:0/2933851497 conn(0x7f1c340b6600 msgr2=0x7f1c340b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/2933851497 shutdown_connections 2024-09-21T07:27:39.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.140+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/2933851497 wait complete. 2024-09-21T07:27:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.142+0000 7f1c3a5ad640 1 Processor -- start 2024-09-21T07:27:39.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.142+0000 7f1c3a5ad640 1 -- start start 2024-09-21T07:27:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34155e00 0x7f1c34141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c3414f6d0 0x7f1c3414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 0x7f1c3414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340af540 con 0x7f1c34150020 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1c340af3c0 con 0x7f1c34155e00 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c3a5ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f1c340af6c0 con 0x7f1c3414f6d0 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c38b4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 0x7f1c3414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c38b4a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 0x7f1c3414dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53710/0 (socket says 172.21.15.105:53710) 2024-09-21T07:27:39.151 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.143+0000 7f1c38b4a640 1 -- 172.21.15.105:0/3524107054 learned_addr learned my addr 172.21.15.105:0/3524107054 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c337fe640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c3414f6d0 0x7f1c3414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c33fff640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34155e00 0x7f1c34141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 -- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c3414f6d0 msgr2=0x7f1c3414fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c3414f6d0 0x7f1c3414fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 -- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34155e00 msgr2=0x7f1c34141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.152 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34155e00 0x7f1c34141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c34142230 con 0x7f1c34150020 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c33fff640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1c34155e00 0x7f1c34141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c337fe640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1c3414f6d0 0x7f1c3414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c38b4a640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 0x7f1c3414dc80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00ef00 tx=0x7f1c2c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c003970 con 0x7f1c34150020 2024-09-21T07:27:39.153 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c2c003c20 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c004280 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c34151b70 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.144+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3414d1a0 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.145+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c34141db0 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.146+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f1c2c01d020 con 0x7f1c34150020 2024-09-21T07:27:39.154 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c30ff9640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1c20078b90 0x7f1c2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f1c2c09b460 con 0x7f1c34150020 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c33fff640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1c20078b90 0x7f1c2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c30ff9640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f1c2007f5d0 0x7f1c20081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c20082130 con 0x7f1c2007f5d0 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c0a2050 con 0x7f1c34150020 2024-09-21T07:27:39.155 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c337fe640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f1c2007f5d0 0x7f1c20081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c33fff640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1c20078b90 0x7f1c2007b0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c24004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.147+0000 7f1c337fe640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f1c2007f5d0 0x7f1c20081a60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.156 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.148+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== osd.4 v2:172.21.15.112:6810/4146883683 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c20082130 con 0x7f1c2007f5d0 2024-09-21T07:27:39.162 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 --> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1c340afb50 con 0x7f1c2007f5d0 2024-09-21T07:27:39.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c30ff9640 1 -- 172.21.15.105:0/3524107054 <== osd.4 v2:172.21.15.112:6810/4146883683 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1c340afb50 con 0x7f1c2007f5d0 2024-09-21T07:27:39.163 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:39.163 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:39.163 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f1c2007f5d0 msgr2=0x7f1c20081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6810/4146883683,v1:172.21.15.112:6811/4146883683] conn(0x7f1c2007f5d0 0x7f1c20081a60 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1c20078b90 msgr2=0x7f1c2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1c20078b90 0x7f1c2007b0a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c24034000 tx=0x7f1c24004730 comp rx=0 tx=0).stop 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 msgr2=0x7f1c3414dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 --2- 172.21.15.105:0/3524107054 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1c34150020 0x7f1c3414dc80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00ef00 tx=0x7f1c2c00c740 comp rx=0 tx=0).stop 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c38b4a640 1 -- 172.21.15.105:0/3524107054 reap_dead start 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 shutdown_connections 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 >> 172.21.15.105:0/3524107054 conn(0x7f1c340b6600 msgr2=0x7f1c3414b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 shutdown_connections 2024-09-21T07:27:39.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.161+0000 7f1c3a5ad640 1 -- 172.21.15.105:0/3524107054 wait complete. 2024-09-21T07:27:39.174 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 -- 172.21.15.105:0/3123506991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940d3880 msgr2=0x7fa7940ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 --2- 172.21.15.105:0/3123506991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940d3880 0x7fa7940ceb50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa7940d2790 tx=0x7fa78c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 -- 172.21.15.105:0/3123506991 shutdown_connections 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 --2- 172.21.15.105:0/3123506991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940d3880 0x7fa7940ceb50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 --2- 172.21.15.105:0/3123506991 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b9790 0x7fa7940d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 --2- 172.21.15.105:0/3123506991 >> v2:172.21.15.112:3300/0 conn(0x7fa7940b3710 0x7fa79406f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 -- 172.21.15.105:0/3123506991 >> 172.21.15.105:0/3123506991 conn(0x7fa79409d850 msgr2=0x7fa79408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 -- 172.21.15.105:0/3123506991 shutdown_connections 2024-09-21T07:27:39.287 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.285+0000 7fa79892b640 1 -- 172.21.15.105:0/3123506991 wait complete. 2024-09-21T07:27:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.287+0000 7fa79892b640 1 Processor -- start 2024-09-21T07:27:39.288 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.287+0000 7fa79892b640 1 -- start start 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b3710 0x7fa794151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7940d3880 0x7fa7941f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940d0cf0 con 0x7fa7940b9790 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7940cf440 con 0x7fa7940d3880 2024-09-21T07:27:39.291 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79892b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fa7940d0e70 con 0x7fa7940b3710 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53734/0 (socket says 172.21.15.105:53734) 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 -- 172.21.15.105:0/1577368058 learned_addr learned my addr 172.21.15.105:0/1577368058 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa792d9d640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7940d3880 0x7fa7941f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa79259c640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b3710 0x7fa794151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 -- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b3710 msgr2=0x7fa794151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b3710 0x7fa794151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 -- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7940d3880 msgr2=0x7fa7941f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7940d3880 0x7fa7941f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.288+0000 7fa791d9b640 1 -- 172.21.15.105:0/1577368058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7941f6220 con 0x7fa7940b9790 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa791d9b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c970 tx=0x7fa78800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788007bb0 con 0x7fa7940b9790 2024-09-21T07:27:39.292 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa788007e40 con 0x7fa7940b9790 2024-09-21T07:27:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7940ce5f0 con 0x7fa7940b9790 2024-09-21T07:27:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788005280 con 0x7fa7940b9790 2024-09-21T07:27:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.289+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7940b54a0 con 0x7fa7940b9790 2024-09-21T07:27:39.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.290+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7940b4fb0 con 0x7fa7940b9790 2024-09-21T07:27:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.291+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7fa788005420 con 0x7fa7940b9790 2024-09-21T07:27:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.291+0000 7fa782ffd640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 0x7fa77407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.291+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7fa78809b1d0 con 0x7fa7940b9790 2024-09-21T07:27:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.291+0000 7fa79259c640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 0x7fa77407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.294 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.292+0000 7fa79259c640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 0x7fa77407b0a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ab20 tx=0x7fa78400a250 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.296+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7941f6220 con 0x7fa7940b9790 2024-09-21T07:27:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.473+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fa7940ce380 con 0x7fa774078b90 2024-09-21T07:27:39.475 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.474+0000 7fa782ffd640 1 -- 172.21.15.105:0/1577368058 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa7940ce380 con 0x7fa774078b90 2024-09-21T07:27:39.476 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 msgr2=0x7fa77407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 0x7fa77407b0a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa78400ab20 tx=0x7fa78400a250 comp rx=0 tx=0).stop 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 msgr2=0x7fa7941f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c970 tx=0x7fa78800ce30 comp rx=0 tx=0).stop 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 shutdown_connections 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7940d3880 0x7fa7941f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fa774078b90 0x7fa77407b0a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7940b9790 0x7fa7941f5ce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 --2- 172.21.15.105:0/1577368058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7940b3710 0x7fa794151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 >> 172.21.15.105:0/1577368058 conn(0x7fa79409d850 msgr2=0x7fa794161980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 shutdown_connections 2024-09-21T07:27:39.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.476+0000 7fa79892b640 1 -- 172.21.15.105:0/1577368058 wait complete. 2024-09-21T07:27:39.489 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T07:27:39.489 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:39.606 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 -- 172.21.15.105:0/4246493162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 msgr2=0x7f608800cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 --2- 172.21.15.105:0/4246493162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f608800cad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6094005fd0 tx=0x7f6088006940 comp rx=0 tx=0).stop 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 -- 172.21.15.105:0/4246493162 shutdown_connections 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 --2- 172.21.15.105:0/4246493162 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f608800cad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 -- 172.21.15.105:0/4246493162 >> 172.21.15.105:0/4246493162 conn(0x7f609c06d3d0 msgr2=0x7f609c0b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 -- 172.21.15.105:0/4246493162 shutdown_connections 2024-09-21T07:27:39.607 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.604+0000 7f60a3b41640 1 -- 172.21.15.105:0/4246493162 wait complete. 2024-09-21T07:27:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.606+0000 7f60a3b41640 1 Processor -- start 2024-09-21T07:27:39.608 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.606+0000 7f60a3b41640 1 -- start start 2024-09-21T07:27:39.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f609c1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f609c0b0de0 0x7f609c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c14ce60 0x7f609c14d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f609c0ce030 con 0x7f6088006410 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f609c0cdeb0 con 0x7f609c14ce60 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.607+0000 7f60a3b41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f609c0ce1b0 con 0x7f609c0b0de0 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f609c1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f609c1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53762/0 (socket says 172.21.15.105:53762) 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 -- 172.21.15.105:0/3447139601 learned_addr learned my addr 172.21.15.105:0/3447139601 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a20de640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c14ce60 0x7f609c14d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a10dc640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f609c0b0de0 0x7f609c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 -- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f609c0b0de0 msgr2=0x7f609c0b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f609c0b0de0 0x7f609c0b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 -- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c14ce60 msgr2=0x7f609c14d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f609c14ce60 0x7f609c14d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.611 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c0b1730 con 0x7f6088006410 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a18dd640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f609c1f4110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6098003e10 tx=0x7f6098001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098015070 con 0x7f6088006410 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6098011840 con 0x7f6088006410 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c1f4650 con 0x7f6088006410 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098011c70 con 0x7f6088006410 2024-09-21T07:27:39.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.608+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c153400 con 0x7f6088006410 2024-09-21T07:27:39.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.609+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f609c0af1b0 con 0x7f6088006410 2024-09-21T07:27:39.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.610+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f6098018070 con 0x7f6088006410 2024-09-21T07:27:39.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.610+0000 7f60927fc640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6080078b90 0x7f608007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f609809bb50 con 0x7f6088006410 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60a10dc640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6080078b90 0x7f608007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60927fc640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f608007f5d0 0x7f6080081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f609800be50 con 0x7f608007f5d0 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f609809bfe0 con 0x7f6088006410 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60a20de640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f608007f5d0 0x7f6080081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60a10dc640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6080078b90 0x7f608007b0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c151050 tx=0x7f608c0067a0 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.611+0000 7f60a20de640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f608007f5d0 0x7f6080081a60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.614 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.612+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== osd.5 v2:172.21.15.112:6826/2151662407 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f609800be50 con 0x7f608007f5d0 2024-09-21T07:27:39.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.624+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 --> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f609c1f2730 con 0x7f608007f5d0 2024-09-21T07:27:39.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60927fc640 1 -- 172.21.15.105:0/3447139601 <== osd.5 v2:172.21.15.112:6826/2151662407 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f609c1f2730 con 0x7f608007f5d0 2024-09-21T07:27:39.627 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:39.627 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:39.627 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f608007f5d0 msgr2=0x7f6080081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6826/2151662407,v1:172.21.15.112:6827/2151662407] conn(0x7f608007f5d0 0x7f6080081a60 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6080078b90 msgr2=0x7f608007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f6080078b90 0x7f608007b0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f609c151050 tx=0x7f608c0067a0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 msgr2=0x7f609c1f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 --2- 172.21.15.105:0/3447139601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6088006410 0x7f609c1f4110 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6098003e10 tx=0x7f6098001630 comp rx=0 tx=0).stop 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a20de640 1 -- 172.21.15.105:0/3447139601 reap_dead start 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 shutdown_connections 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 >> 172.21.15.105:0/3447139601 conn(0x7f609c06d3d0 msgr2=0x7f609c073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 shutdown_connections 2024-09-21T07:27:39.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.625+0000 7f60a3b41640 1 -- 172.21.15.105:0/3447139601 wait complete. 2024-09-21T07:27:39.638 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 -- 172.21.15.105:0/791835210 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05680b2ce0 msgr2=0x7f05680b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 --2- 172.21.15.105:0/791835210 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05680b2ce0 0x7f05680b30f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f055c009830 tx=0x7f055c02a510 comp rx=0 tx=0).stop 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 -- 172.21.15.105:0/791835210 shutdown_connections 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 --2- 172.21.15.105:0/791835210 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05680b2ce0 0x7f05680b30f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 --2- 172.21.15.105:0/791835210 >> v2:172.21.15.112:3300/0 conn(0x7f05680ce870 0x7f056815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 -- 172.21.15.105:0/791835210 >> 172.21.15.105:0/791835210 conn(0x7f05680cdce0 msgr2=0x7f05680ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.749+0000 7f056cc7e640 1 -- 172.21.15.105:0/791835210 shutdown_connections 2024-09-21T07:27:39.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.750+0000 7f056cc7e640 1 -- 172.21.15.105:0/791835210 wait complete. 2024-09-21T07:27:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.751+0000 7f056cc7e640 1 Processor -- start 2024-09-21T07:27:39.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.751+0000 7f056cc7e640 1 -- start start 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 0x7f05681f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815f2a0 0x7f0568149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b5d90 con 0x7f05680b2ce0 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05680b40b0 con 0x7f05680ce870 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056cc7e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f056815fdc0 con 0x7f056815f2a0 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056659c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53774/0 (socket says 172.21.15.105:53774) 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056659c640 1 -- 172.21.15.105:0/1001180838 learned_addr learned my addr 172.21.15.105:0/1001180838 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0566d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815f2a0 0x7f0568149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.756 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 0x7f05681f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 -- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815f2a0 msgr2=0x7f0568149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815f2a0 0x7f0568149fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 -- 172.21.15.105:0/1001180838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 msgr2=0x7f056815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 -- 172.21.15.105:0/1001180838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05681f49d0 con 0x7f05680ce870 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f056659c640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.753+0000 7f0565d9b640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 0x7f05681f4490 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f055c031000 tx=0x7f055c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.754+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c005220 con 0x7f05680ce870 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.754+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f055c0054b0 con 0x7f05680ce870 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.754+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05680b2810 con 0x7f05680ce870 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.754+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c004a00 con 0x7f05680ce870 2024-09-21T07:27:39.757 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.754+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056815d260 con 0x7f05680ce870 2024-09-21T07:27:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.755+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05680b2580 con 0x7f05680ce870 2024-09-21T07:27:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.755+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f055c004ba0 con 0x7f05680ce870 2024-09-21T07:27:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.756+0000 7f0556ffd640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 0x7f054807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.756+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f055c0bc0a0 con 0x7f05680ce870 2024-09-21T07:27:39.758 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.756+0000 7f056659c640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 0x7f054807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:39.759 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.756+0000 7f056659c640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 0x7f054807b0d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05580099c0 tx=0x7f0558009500 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:39.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.761+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f055c03f030 con 0x7f05680ce870 2024-09-21T07:27:39.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.937+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f05680b30f0 con 0x7f0548078bc0 2024-09-21T07:27:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.937+0000 7f0556ffd640 1 -- 172.21.15.105:0/1001180838 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05680b30f0 con 0x7f0548078bc0 2024-09-21T07:27:39.939 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T07:27:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 msgr2=0x7f054807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 0x7f054807b0d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05580099c0 tx=0x7f0558009500 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 msgr2=0x7f05681f4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 0x7f05681f4490 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f055c031000 tx=0x7f055c0040c0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 shutdown_connections 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05680ce870 0x7f05681f4490 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f0548078bc0 0x7f054807b0d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f056815f2a0 0x7f0568149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 --2- 172.21.15.105:0/1001180838 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f05680b2ce0 0x7f056815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 >> 172.21.15.105:0/1001180838 conn(0x7f05680cdce0 msgr2=0x7f05680b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 shutdown_connections 2024-09-21T07:27:39.942 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:39.939+0000 7f056cc7e640 1 -- 172.21.15.105:0/1001180838 wait complete. 2024-09-21T07:27:39.952 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T07:27:39.953 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 -- 172.21.15.105:0/1458900438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3120160b30 msgr2=0x7f31200cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 --2- 172.21.15.105:0/1458900438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3120160b30 0x7f31200cd410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3120073790 tx=0x7f311002a410 comp rx=0 tx=0).stop 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 -- 172.21.15.105:0/1458900438 shutdown_connections 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 --2- 172.21.15.105:0/1458900438 >> v2:172.21.15.112:3300/0 conn(0x7f31200b5dd0 0x7f31200b61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 --2- 172.21.15.105:0/1458900438 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3120160b30 0x7f31200cd410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.070 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 -- 172.21.15.105:0/1458900438 >> 172.21.15.105:0/1458900438 conn(0x7f312006cf80 msgr2=0x7f312006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 -- 172.21.15.105:0/1458900438 shutdown_connections 2024-09-21T07:27:40.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.068+0000 7f312668b640 1 -- 172.21.15.105:0/1458900438 wait complete. 2024-09-21T07:27:40.071 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.070+0000 7f312668b640 1 Processor -- start 2024-09-21T07:27:40.072 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.070+0000 7f312668b640 1 -- start start 2024-09-21T07:27:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3120070930 0x7f31200af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 0x7f31200ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3120160b30 0x7f312006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.074 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f3120161130 con 0x7f31200b5dd0 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31200cdb60 con 0x7f3120160b30 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f312668b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f31200ce9e0 con 0x7f3120070930 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 0x7f31200ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f311f7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 0x7f31200ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:53792/0 (socket says 172.21.15.105:53792) 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f311f7fe640 1 -- 172.21.15.105:0/2086995828 learned_addr learned my addr 172.21.15.105:0/2086995828 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f3124c28640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3120160b30 0x7f312006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.071+0000 7f311ffff640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3120070930 0x7f31200af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 -- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3120070930 msgr2=0x7f31200af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3120070930 0x7f31200af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 -- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3120160b30 msgr2=0x7f312006f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3120160b30 0x7f312006f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31201481d0 con 0x7f31200b5dd0 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311ffff640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3120070930 0x7f31200af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:27:40.075 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311f7fe640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 0x7f31200ad590 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f311400b410 tx=0x7f311400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114015020 con 0x7f31200b5dd0 2024-09-21T07:27:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3114011940 con 0x7f31200b5dd0 2024-09-21T07:27:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201486d0 con 0x7f31200b5dd0 2024-09-21T07:27:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114011d90 con 0x7f31200b5dd0 2024-09-21T07:27:40.076 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.072+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3120148c90 con 0x7f31200b5dd0 2024-09-21T07:27:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.073+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31201f6ab0 con 0x7f31200b5dd0 2024-09-21T07:27:40.077 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.074+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f3114018020 con 0x7f31200b5dd0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.074+0000 7f311cff9640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f310c078b90 0x7f310c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.074+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f311409b2e0 con 0x7f31200b5dd0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f311cff9640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f310c07f5d0 0x7f310c081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f311401deb0 con 0x7f310c07f5d0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f311409b770 con 0x7f31200b5dd0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f311ffff640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f310c078b90 0x7f310c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f3124c28640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f310c07f5d0 0x7f310c081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f311ffff640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f310c078b90 0x7f310c07b0a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3110034000 tx=0x7f3110004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.075+0000 7f3124c28640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f310c07f5d0 0x7f310c081a60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.078 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.076+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== osd.6 v2:172.21.15.112:6818/2596982695 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f311401deb0 con 0x7f310c07f5d0 2024-09-21T07:27:40.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.088+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 --> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3120148480 con 0x7f310c07f5d0 2024-09-21T07:27:40.090 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f311cff9640 1 -- 172.21.15.105:0/2086995828 <== osd.6 v2:172.21.15.112:6818/2596982695 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3120148480 con 0x7f310c07f5d0 2024-09-21T07:27:40.090 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:40.091 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:40.091 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:40.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f310c07f5d0 msgr2=0x7f310c081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.091 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6818/2596982695,v1:172.21.15.112:6819/2596982695] conn(0x7f310c07f5d0 0x7f310c081a60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f310c078b90 msgr2=0x7f310c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f310c078b90 0x7f310c07b0a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3110034000 tx=0x7f3110004730 comp rx=0 tx=0).stop 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 msgr2=0x7f31200ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 --2- 172.21.15.105:0/2086995828 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f31200b5dd0 0x7f31200ad590 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f311400b410 tx=0x7f311400b8d0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f3124c28640 1 -- 172.21.15.105:0/2086995828 reap_dead start 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 shutdown_connections 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 >> 172.21.15.105:0/2086995828 conn(0x7f312006cf80 msgr2=0x7f312015d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 shutdown_connections 2024-09-21T07:27:40.092 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.089+0000 7f312668b640 1 -- 172.21.15.105:0/2086995828 wait complete. 2024-09-21T07:27:40.102 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 -- 172.21.15.105:0/2243502076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d3020 msgr2=0x7f37e80b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 --2- 172.21.15.105:0/2243502076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d3020 0x7f37e80b3cb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f37e808d170 tx=0x7f37d802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 -- 172.21.15.105:0/2243502076 shutdown_connections 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 --2- 172.21.15.105:0/2243502076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 0x7f37e80d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 --2- 172.21.15.105:0/2243502076 >> v2:172.21.15.112:3300/0 conn(0x7f37e80d4460 0x7f37e80d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 --2- 172.21.15.105:0/2243502076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d3020 0x7f37e80b3cb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 -- 172.21.15.105:0/2243502076 >> 172.21.15.105:0/2243502076 conn(0x7f37e808bfb0 msgr2=0x7f37e809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 -- 172.21.15.105:0/2243502076 shutdown_connections 2024-09-21T07:27:40.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.215+0000 7f37ed78b640 1 -- 172.21.15.105:0/2243502076 wait complete. 2024-09-21T07:27:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.217+0000 7f37ed78b640 1 Processor -- start 2024-09-21T07:27:40.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.217+0000 7f37ed78b640 1 -- start start 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 0x7f37e815ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e80d3020 0x7f37e80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80b4aa0 con 0x7f37e80d4460 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80b4920 con 0x7f37e80d3020 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37ed78b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f37e80d1910 con 0x7f37e80d0fc0 2024-09-21T07:27:40.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37e77fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34330/0 (socket says 172.21.15.105:34330) 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37e77fe640 1 -- 172.21.15.105:0/3060492134 learned_addr learned my addr 172.21.15.105:0/3060492134 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37e6ffd640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 0x7f37e815ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37de7fc640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e80d3020 0x7f37e80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.218+0000 7f37e77fe640 1 -- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 msgr2=0x7f37e815ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37e77fe640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 0x7f37e815ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37e77fe640 1 -- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e80d3020 msgr2=0x7f37e80cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37e77fe640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e80d3020 0x7f37e80cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37e77fe640 1 -- 172.21.15.105:0/3060492134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e80cc870 con 0x7f37e80d4460 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37e77fe640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37e000ba10 tx=0x7f37e000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0019070 con 0x7f37e80d4460 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37e000cc50 con 0x7f37e80d4460 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e80ccb10 con 0x7f37e80d4460 2024-09-21T07:27:40.222 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00146a0 con 0x7f37e80d4460 2024-09-21T07:27:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.219+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e80cd0d0 con 0x7f37e80d4460 2024-09-21T07:27:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.220+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e80b3cb0 con 0x7f37e80d4460 2024-09-21T07:27:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.221+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f37e001c070 con 0x7f37e80d4460 2024-09-21T07:27:40.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.221+0000 7f37dffff640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.221+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f37e009aa50 con 0x7f37e80d4460 2024-09-21T07:27:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.221+0000 7f37e6ffd640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.224 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.222+0000 7f37e6ffd640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 0x7f37c807b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37d8034000 tx=0x7f37d8004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.227 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.226+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37e0062750 con 0x7f37e80d4460 2024-09-21T07:27:40.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.402+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f37e80d13d0 con 0x7f37c8078b90 2024-09-21T07:27:40.404 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.403+0000 7f37dffff640 1 -- 172.21.15.105:0/3060492134 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f37e80d13d0 con 0x7f37c8078b90 2024-09-21T07:27:40.404 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 msgr2=0x7f37c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 0x7f37c807b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f37d8034000 tx=0x7f37d8004730 comp rx=0 tx=0).stop 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 msgr2=0x7f37e81f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f37e000ba10 tx=0x7f37e000bed0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 shutdown_connections 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37e80d3020 0x7f37e80cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f37c8078b90 0x7f37c807b0a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f37e80d4460 0x7f37e81f8ba0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 --2- 172.21.15.105:0/3060492134 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37e80d0fc0 0x7f37e815ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 >> 172.21.15.105:0/3060492134 conn(0x7f37e808bfb0 msgr2=0x7f37e806f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 shutdown_connections 2024-09-21T07:27:40.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.405+0000 7f37ed78b640 1 -- 172.21.15.105:0/3060492134 wait complete. 2024-09-21T07:27:40.417 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T07:27:40.418 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:27:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 -- 172.21.15.105:0/3512138158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200c59a0 msgr2=0x7f5d200d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 --2- 172.21.15.105:0/3512138158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200c59a0 0x7f5d200d0180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5d10009860 tx=0x7f5d1002a420 comp rx=0 tx=0).stop 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 -- 172.21.15.105:0/3512138158 shutdown_connections 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 --2- 172.21.15.105:0/3512138158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d200b3340 0x7f5d200cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 --2- 172.21.15.105:0/3512138158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200c59a0 0x7f5d200d0180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 -- 172.21.15.105:0/3512138158 >> 172.21.15.105:0/3512138158 conn(0x7f5d200ced90 msgr2=0x7f5d200cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 -- 172.21.15.105:0/3512138158 shutdown_connections 2024-09-21T07:27:40.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.533+0000 7f5d24d72640 1 -- 172.21.15.105:0/3512138158 wait complete. 2024-09-21T07:27:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.535+0000 7f5d24d72640 1 Processor -- start 2024-09-21T07:27:40.537 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.535+0000 7f5d24d72640 1 -- start start 2024-09-21T07:27:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200b2f30 0x7f5d2015feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 0x7f5d200caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d200c59a0 0x7f5d2014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f5d200b1a90 con 0x7f5d200b3340 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2009bc40 con 0x7f5d200b2f30 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f5d200b1c10 con 0x7f5d200c59a0 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 0x7f5d200caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 0x7f5d200caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34342/0 (socket says 172.21.15.105:34342) 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 -- 172.21.15.105:0/862851743 learned_addr learned my addr 172.21.15.105:0/862851743 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1e59c640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200b2f30 0x7f5d2015feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1ed9d640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d200c59a0 0x7f5d2014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 -- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d200c59a0 msgr2=0x7f5d2014e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5d200c59a0 0x7f5d2014e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 -- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200b2f30 msgr2=0x7f5d2015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.540 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d200b2f30 0x7f5d2015feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d200c66c0 con 0x7f5d200b3340 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d1dd9b640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 0x7f5d200caa90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c3c0 tx=0x7f5d1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14011790 con 0x7f5d200b3340 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d14017070 con 0x7f5d200b3340 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.537+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d200c69a0 con 0x7f5d200b3340 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.538+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14011ce0 con 0x7f5d200b3340 2024-09-21T07:27:40.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.538+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d200c6f60 con 0x7f5d200b3340 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.538+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d2014f380 con 0x7f5d200b3340 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.539+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f5d1401c460 con 0x7f5d200b3340 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d0effd640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5d00078b90 0x7f5d0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f5d1409b4d0 con 0x7f5d200b3340 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d0effd640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f5d0007f5d0 0x7f5d00081a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d14023e70 con 0x7f5d0007f5d0 2024-09-21T07:27:40.542 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5d1409b960 con 0x7f5d200b3340 2024-09-21T07:27:40.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d1e59c640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5d00078b90 0x7f5d0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.540+0000 7f5d1ed9d640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f5d0007f5d0 0x7f5d00081a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.541+0000 7f5d1e59c640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5d00078b90 0x7f5d0007b0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d10034000 tx=0x7f5d10004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.541+0000 7f5d1ed9d640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f5d0007f5d0 0x7f5d00081a60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.542+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== osd.7 v2:172.21.15.112:6802/3904329758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d14023e70 con 0x7f5d0007f5d0 2024-09-21T07:27:40.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.554+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 --> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5d2014f130 con 0x7f5d0007f5d0 2024-09-21T07:27:40.556 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d0effd640 1 -- 172.21.15.105:0/862851743 <== osd.7 v2:172.21.15.112:6802/3904329758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5d2014f130 con 0x7f5d0007f5d0 2024-09-21T07:27:40.557 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-09-21T07:27:40.557 INFO:teuthology.orchestra.run.smithi105.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:27:40.557 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f5d0007f5d0 msgr2=0x7f5d00081a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6802/3904329758,v1:172.21.15.112:6803/3904329758] conn(0x7f5d0007f5d0 0x7f5d00081a60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5d00078b90 msgr2=0x7f5d0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f5d00078b90 0x7f5d0007b0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d10034000 tx=0x7f5d10004730 comp rx=0 tx=0).stop 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 msgr2=0x7f5d200caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 --2- 172.21.15.105:0/862851743 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5d200b3340 0x7f5d200caa90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c3c0 tx=0x7f5d1400c880 comp rx=0 tx=0).stop 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d1ed9d640 1 -- 172.21.15.105:0/862851743 reap_dead start 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 shutdown_connections 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 >> 172.21.15.105:0/862851743 conn(0x7f5d200ced90 msgr2=0x7f5d200b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.558 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 shutdown_connections 2024-09-21T07:27:40.559 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.555+0000 7f5d24d72640 1 -- 172.21.15.105:0/862851743 wait complete. 2024-09-21T07:27:40.567 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 -- 172.21.15.105:0/3178086674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f00d16f0 msgr2=0x7fe7f00b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/3178086674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f00d16f0 0x7fe7f00b66d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00a2390 tx=0x7fe7e402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 -- 172.21.15.105:0/3178086674 shutdown_connections 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/3178086674 >> v2:172.21.15.112:3300/0 conn(0x7fe7f006e010 0x7fe7f00b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/3178086674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f00d16f0 0x7fe7f00b66d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 -- 172.21.15.105:0/3178086674 >> 172.21.15.105:0/3178086674 conn(0x7fe7f009d810 msgr2=0x7fe7f008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.681 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 -- 172.21.15.105:0/3178086674 shutdown_connections 2024-09-21T07:27:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.679+0000 7fe7f59a8640 1 -- 172.21.15.105:0/3178086674 wait complete. 2024-09-21T07:27:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.681+0000 7fe7f59a8640 1 Processor -- start 2024-09-21T07:27:40.682 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.681+0000 7fe7f59a8640 1 -- start start 2024-09-21T07:27:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.685 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f00b3800 0x7fe7f01f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f00d16f0 0x7fe7f01f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00b2350 con 0x7fe7f006e010 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f00b21d0 con 0x7fe7f00d16f0 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7f59a8640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f00b24d0 con 0x7fe7f00b3800 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7eeffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34354/0 (socket says 172.21.15.105:34354) 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7eeffd640 1 -- 172.21.15.105:0/1632733718 learned_addr learned my addr 172.21.15.105:0/1632733718 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7ee7fc640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f00b3800 0x7fe7f01f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.682+0000 7fe7ef7fe640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f00d16f0 0x7fe7f01f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 -- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f00b3800 msgr2=0x7fe7f01f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f00b3800 0x7fe7f01f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.686 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 -- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f00d16f0 msgr2=0x7fe7f01f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f00d16f0 0x7fe7f01f98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 -- 172.21.15.105:0/1632733718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f00cd240 con 0x7fe7f006e010 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7eeffd640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c3c0 tx=0x7fe7e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7f006e010 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7e00053c0 con 0x7fe7f006e010 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f01fa2b0 con 0x7fe7f006e010 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0003880 con 0x7fe7f006e010 2024-09-21T07:27:40.687 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.683+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f015e010 con 0x7fe7f006e010 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.684+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f00b29f0 con 0x7fe7f006e010 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.685+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7fe7e001a070 con 0x7fe7f006e010 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.685+0000 7fe7d7fff640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 0x7fe7dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.685+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7fe7e009a1d0 con 0x7fe7f006e010 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.685+0000 7fe7ee7fc640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 0x7fe7dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:40.688 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.686+0000 7fe7ee7fc640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 0x7fe7dc07b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4033000 tx=0x7fe7e4004730 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:40.691 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.690+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe7e0061e50 con 0x7fe7f006e010 2024-09-21T07:27:40.871 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.869+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe7f006a170 con 0x7fe7dc078b90 2024-09-21T07:27:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.870+0000 7fe7d7fff640 1 -- 172.21.15.105:0/1632733718 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe7f006a170 con 0x7fe7dc078b90 2024-09-21T07:27:40.872 INFO:teuthology.orchestra.run.smithi105.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T07:27:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 msgr2=0x7fe7dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 0x7fe7dc07b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4033000 tx=0x7fe7e4004730 comp rx=0 tx=0).stop 2024-09-21T07:27:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 msgr2=0x7fe7f00ccd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000c3c0 tx=0x7fe7e000c880 comp rx=0 tx=0).stop 2024-09-21T07:27:40.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 shutdown_connections 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7f00d16f0 0x7fe7f01f98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7fe7dc078b90 0x7fe7dc07b0a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7f00b3800 0x7fe7f01f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 --2- 172.21.15.105:0/1632733718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe7f006e010 0x7fe7f00ccd00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 >> 172.21.15.105:0/1632733718 conn(0x7fe7f009d810 msgr2=0x7fe7f0161c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 shutdown_connections 2024-09-21T07:27:40.876 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.872+0000 7fe7f59a8640 1 -- 172.21.15.105:0/1632733718 wait complete. 2024-09-21T07:27:40.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 -- 172.21.15.105:0/1493372644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2630004d40 msgr2=0x7f2630005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 --2- 172.21.15.105:0/1493372644 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2630004d40 0x7f2630005210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f264000baf0 tx=0x7f2630005830 comp rx=0 tx=0).stop 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f2647fff640 1 -- 172.21.15.105:0/1493372644 reap_dead start 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 -- 172.21.15.105:0/1493372644 shutdown_connections 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 -- 172.21.15.105:0/1493372644 >> 172.21.15.105:0/1493372644 conn(0x7f26480cedb0 msgr2=0x7f26480cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 -- 172.21.15.105:0/1493372644 shutdown_connections 2024-09-21T07:27:40.999 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.997+0000 7f264df6e640 1 -- 172.21.15.105:0/1493372644 wait complete. 2024-09-21T07:27:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.998+0000 7f264df6e640 1 Processor -- start 2024-09-21T07:27:41.000 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:40.999+0000 7f264df6e640 1 -- start start 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26480cbb40 0x7f26480c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26480c8d60 0x7f264806e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f26480cebe0 con 0x7f26481518d0 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f26480cea60 con 0x7f26480c8d60 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f264df6e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f26480d0310 con 0x7f26480cbb40 2024-09-21T07:27:41.003 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:34384/0 (socket says 172.21.15.105:34384) 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 -- 172.21.15.105:0/56738144 learned_addr learned my addr 172.21.15.105:0/56738144 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f2647fff640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26480c8d60 0x7f264806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f2646ffd640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26480cbb40 0x7f26480c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 -- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26480cbb40 msgr2=0x7f26480c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26480cbb40 0x7f26480c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 -- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26480c8d60 msgr2=0x7f264806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26480c8d60 0x7f264806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.000+0000 7f26477fe640 1 -- 172.21.15.105:0/56738144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648151ce0 con 0x7f26481518d0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f26477fe640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2638001be0 tx=0x7f2638006860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638015070 con 0x7f26481518d0 2024-09-21T07:27:41.004 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f263800fe00 con 0x7f26481518d0 2024-09-21T07:27:41.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26480ca560 con 0x7f26481518d0 2024-09-21T07:27:41.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638011690 con 0x7f26481518d0 2024-09-21T07:27:41.005 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.001+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26480cab20 con 0x7f26481518d0 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.002+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26480d0e40 con 0x7f26481518d0 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.003+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f2638011830 con 0x7f26481518d0 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.003+0000 7f262ffff640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.003+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f263809bea0 con 0x7f26481518d0 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.003+0000 7f2646ffd640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:27:41.006 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.004+0000 7f2646ffd640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 0x7f263407b0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2648072bb0 tx=0x7f263c00b260 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:27:41.009 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.008+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2638063cc0 con 0x7f26481518d0 2024-09-21T07:27:41.184 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.182+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f26480b3c00 con 0x7f2634078b90 2024-09-21T07:27:41.185 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.183+0000 7f262ffff640 1 -- 172.21.15.105:0/56738144 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13485 (secure 0 0 0) 0x7f26480b3c00 con 0x7f2634078b90 2024-09-21T07:27:41.185 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:27:41.186 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 msgr2=0x7f263407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 0x7f263407b0a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2648072bb0 tx=0x7f263c00b260 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 msgr2=0x7f26480cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2638001be0 tx=0x7f2638006860 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 shutdown_connections 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f26480c8d60 0x7f264806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2634078b90 0x7f263407b0a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f26480cbb40 0x7f26480c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 --2- 172.21.15.105:0/56738144 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f26481518d0 0x7f26480cb600 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:27:41.188 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 >> 172.21.15.105:0/56738144 conn(0x7f26480cedb0 msgr2=0x7f26480cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:27:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 shutdown_connections 2024-09-21T07:27:41.189 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:27:41.186+0000 7f264df6e640 1 -- 172.21.15.105:0/56738144 wait complete. 2024-09-21T07:27:41.199 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-21T07:27:39.489979+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225748,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748972,"statfs":{"total":773094113280,"available":772862947328,"internally_reserved":0,"allocated":5480448,"data_stored":3085568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-732314,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-357,"num_read_kb":-195,"num_write":-344,"num_write_kb":-2903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001578"},"pg_stats":[{"pgid":"1.0","version":"45'86","reported_seq":207,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:27:32.913860+0000","last_change":"2024-09-21T07:26:06.812059+0000","last_active":"2024-09-21T07:27:32.913860+0000","last_peered":"2024-09-21T07:27:32.913860+0000","last_clean":"2024-09-21T07:27:32.913860+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:32.913860+0000","last_undegraded":"2024-09-21T07:27:32.913860+0000","last_fullsized":"2024-09-21T07:27:32.913860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:26:04.884760+0000","last_clean_scrub_stamp":"2024-09-21T07:26:04.884760+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:21.410186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":828472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1130496,"data_stored":828472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:27:41.201 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T07:26:04.884760+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=26, tm_sec=4, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=7, tm_min=27, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T07:27:41.201 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T07:27:42.582 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:27:42.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:28:01.202 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:28:01.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 -- 172.21.15.105:0/2380467147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 msgr2=0x7f421c0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.329 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2380467147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0d0380 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f420c005fd0 tx=0x7f420c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 -- 172.21.15.105:0/2380467147 shutdown_connections 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2380467147 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cdbb0 0x7f421c167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2380467147 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0d0380 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 -- 172.21.15.105:0/2380467147 >> 172.21.15.105:0/2380467147 conn(0x7f421c0cee20 msgr2=0x7f421c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 -- 172.21.15.105:0/2380467147 shutdown_connections 2024-09-21T07:28:01.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.327+0000 7f4220b8b640 1 -- 172.21.15.105:0/2380467147 wait complete. 2024-09-21T07:28:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.329+0000 7f4220b8b640 1 Processor -- start 2024-09-21T07:28:01.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.329+0000 7f4220b8b640 1 -- start start 2024-09-21T07:28:01.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cd090 0x7f421c1e9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0cdbb0 0x7f421c0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f421c0b77d0 con 0x7f421c0cff70 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f421c074af0 con 0x7f421c0cdbb0 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4220b8b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f421c0b7950 con 0x7f421c0cd090 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40544/0 (socket says 172.21.15.105:40544) 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 -- 172.21.15.105:0/2206696873 learned_addr learned my addr 172.21.15.105:0/2206696873 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421a59c640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cd090 0x7f421c1e9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f4219d9b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0cdbb0 0x7f421c0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 -- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cd090 msgr2=0x7f421c1e9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cd090 0x7f421c1e9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 -- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0cdbb0 msgr2=0x7f421c0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0cdbb0 0x7f421c0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 -- 172.21.15.105:0/2206696873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c1f3ab0 con 0x7f421c0cff70 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f421ad9d640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f421400e6a0 tx=0x7f421400eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421400cab0 con 0x7f421c0cff70 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.331+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f421400cc50 con 0x7f421c0cff70 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.332+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c0b1d20 con 0x7f421c0cff70 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.332+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214005280 con 0x7f421c0cff70 2024-09-21T07:28:01.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.332+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c1f4030 con 0x7f421c0cff70 2024-09-21T07:28:01.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.333+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421c069bb0 con 0x7f421c0cff70 2024-09-21T07:28:01.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.333+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f4214005420 con 0x7f421c0cff70 2024-09-21T07:28:01.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.334+0000 7f420affd640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 0x7f41fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.334+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f421409b690 con 0x7f421c0cff70 2024-09-21T07:28:01.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.335+0000 7f421a59c640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 0x7f41fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.335+0000 7f421a59c640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 0x7f41fc07b0a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f420c034000 tx=0x7f420c004ab0 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.339+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f421c069bb0 con 0x7f421c0cff70 2024-09-21T07:28:01.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.514+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 --> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f421c1e02c0 con 0x7f41fc078b90 2024-09-21T07:28:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.515+0000 7f420affd640 1 -- 172.21.15.105:0/2206696873 <== mgr.5364 v2:172.21.15.112:6800/440752644 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13349 (secure 0 0 0) 0x7f420c003ce0 con 0x7f41fc078b90 2024-09-21T07:28:01.517 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:28:01.517 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-09-21T07:28:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 msgr2=0x7f41fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 0x7f41fc07b0a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f420c034000 tx=0x7f420c004ab0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.519 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 msgr2=0x7f421c0b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f421400e6a0 tx=0x7f421400eb60 comp rx=0 tx=0).stop 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 shutdown_connections 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f421c0cdbb0 0x7f421c0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f41fc078b90 0x7f41fc07b0a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f421c0cff70 0x7f421c0b16c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 --2- 172.21.15.105:0/2206696873 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f421c0cd090 0x7f421c1e9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 >> 172.21.15.105:0/2206696873 conn(0x7f421c0cee20 msgr2=0x7f421c1e5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 shutdown_connections 2024-09-21T07:28:01.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.517+0000 7f4220b8b640 1 -- 172.21.15.105:0/2206696873 wait complete. 2024-09-21T07:28:01.531 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-21T07:28:01.495102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225740,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748980,"statfs":{"total":773094113280,"available":772862955520,"internally_reserved":0,"allocated":5480448,"data_stored":3085568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002562"},"pg_stats":[{"pgid":"1.0","version":"46'87","reported_seq":216,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:27:41.266673+0000","last_change":"2024-09-21T07:27:41.266673+0000","last_active":"2024-09-21T07:27:41.266673+0000","last_peered":"2024-09-21T07:27:41.266673+0000","last_clean":"2024-09-21T07:27:41.266673+0000","last_became_active":"2024-09-21T07:26:06.810714+0000","last_became_peered":"2024-09-21T07:26:06.810714+0000","last_unstale":"2024-09-21T07:27:41.266673+0000","last_undegraded":"2024-09-21T07:27:41.266673+0000","last_fullsized":"2024-09-21T07:27:41.266673+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'87","last_scrub_stamp":"2024-09-21T07:27:41.266573+0000","last_deep_scrub":"46'87","last_deep_scrub_stamp":"2024-09-21T07:27:41.266573+0000","last_clean_scrub_stamp":"2024-09-21T07:27:41.266573+0000","objects_scrubbed":2,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:51.902938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":407,"num_write":125,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87,"ondisk_log_size":87,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":828472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509466,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509467,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31712,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":238104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":828472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:28:01.531 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 -- 172.21.15.105:0/4077413931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880cf1b0 msgr2=0x7f22880cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 --2- 172.21.15.105:0/4077413931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880cf1b0 0x7f22880cf5c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2280005fd0 tx=0x7f228002a820 comp rx=0 tx=0).stop 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 -- 172.21.15.105:0/4077413931 shutdown_connections 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 --2- 172.21.15.105:0/4077413931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880cf1b0 0x7f22880cf5c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 --2- 172.21.15.105:0/4077413931 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880d3020 0x7f22880b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 -- 172.21.15.105:0/4077413931 >> 172.21.15.105:0/4077413931 conn(0x7f228808bfb0 msgr2=0x7f228809d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 -- 172.21.15.105:0/4077413931 shutdown_connections 2024-09-21T07:28:01.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.644+0000 7f228dfda640 1 -- 172.21.15.105:0/4077413931 wait complete. 2024-09-21T07:28:01.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.646+0000 7f228dfda640 1 Processor -- start 2024-09-21T07:28:01.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.646+0000 7f228dfda640 1 -- start start 2024-09-21T07:28:01.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880cf1b0 0x7f22880b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 0x7f2288157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f22880cfb00 con 0x7f22880d4460 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22880b4b30 con 0x7f22880d3020 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f228dfda640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f22880cfc80 con 0x7f22880cf1b0 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f2287fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f2287fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40562/0 (socket says 172.21.15.105:40562) 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.647+0000 7f2287fff640 1 -- 172.21.15.105:0/1549822096 learned_addr learned my addr 172.21.15.105:0/1549822096 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f22877fe640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880cf1b0 0x7f22880b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f227ffff640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 0x7f2288157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 -- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880cf1b0 msgr2=0x7f22880b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880cf1b0 0x7f22880b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 -- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 msgr2=0x7f2288157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 0x7f2288157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 -- 172.21.15.105:0/1549822096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22880b3da0 con 0x7f22880d4460 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f227ffff640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 0x7f2288157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2287fff640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f2280032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280037040 con 0x7f22880d4460 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2280042070 con 0x7f22880d4460 2024-09-21T07:28:01.653 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881f9040 con 0x7f22880d4460 2024-09-21T07:28:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800042a0 con 0x7f22880d4460 2024-09-21T07:28:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.648+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881f9600 con 0x7f22880d4460 2024-09-21T07:28:01.654 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.649+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22880d0100 con 0x7f22880d4460 2024-09-21T07:28:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.650+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f2280004440 con 0x7f22880d4460 2024-09-21T07:28:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.650+0000 7f2284ff9640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 0x7f226807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.651+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f22800bbd80 con 0x7f22880d4460 2024-09-21T07:28:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.651+0000 7f22877fe640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 0x7f226807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.655 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.651+0000 7f22877fe640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 0x7f226807b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2278005860 tx=0x7f227800c440 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.655+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2280084050 con 0x7f22880d4460 2024-09-21T07:28:01.843 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.841+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f22880b3b30 con 0x7f22880d4460 2024-09-21T07:28:01.847 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.845+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v36)=0 v36) ==== 123+0+0 (secure 0 0 0) 0x7f2280083950 con 0x7f22880d4460 2024-09-21T07:28:01.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.846+0000 7f2284ff9640 1 -- 172.21.15.105:0/1549822096 <== mon.0 v2:172.21.15.105:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f2280032540 con 0x7f22880d4460 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 msgr2=0x7f226807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 0x7f226807b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2278005860 tx=0x7f227800c440 comp rx=0 tx=0).stop 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 msgr2=0x7f228815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f2280032d10 comp rx=0 tx=0).stop 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 shutdown_connections 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f22880d3020 0x7f2288157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f2268078b90 0x7f226807b0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f22880d4460 0x7f228815ceb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 --2- 172.21.15.105:0/1549822096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f22880cf1b0 0x7f22880b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 >> 172.21.15.105:0/1549822096 conn(0x7f228808bfb0 msgr2=0x7f22880b8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:01.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 shutdown_connections 2024-09-21T07:28:01.852 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.848+0000 7f228dfda640 1 -- 172.21.15.105:0/1549822096 wait complete. 2024-09-21T07:28:01.861 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:28:01.862 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:28:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 -- 172.21.15.105:0/255342779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 msgr2=0x7f1720003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 --2- 172.21.15.105:0/255342779 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f1720003ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1730009b40 tx=0x7f1720005830 comp rx=0 tx=0).stop 2024-09-21T07:28:01.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f1739960640 1 -- 172.21.15.105:0/255342779 reap_dead start 2024-09-21T07:28:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 -- 172.21.15.105:0/255342779 shutdown_connections 2024-09-21T07:28:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 -- 172.21.15.105:0/255342779 >> 172.21.15.105:0/255342779 conn(0x7f173408bfb0 msgr2=0x7f173409d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 -- 172.21.15.105:0/255342779 shutdown_connections 2024-09-21T07:28:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.976+0000 7f173b3c3640 1 -- 172.21.15.105:0/255342779 wait complete. 2024-09-21T07:28:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.979+0000 7f173b3c3640 1 Processor -- start 2024-09-21T07:28:01.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.979+0000 7f173b3c3640 1 -- start start 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17340cc6a0 0x7f1734160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734160750 0x7f17341f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 -- 0x7f17340b40c0 con 0x7f1720003a30 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17340b3f40 con 0x7f1734160750 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173b3c3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 -- 0x7f17340b4240 con 0x7f17340cc6a0 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:40574/0 (socket says 172.21.15.105:40574) 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 -- 172.21.15.105:0/3140459333 learned_addr learned my addr 172.21.15.105:0/3140459333 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f1739960640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734160750 0x7f17341f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173895e640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17340cc6a0 0x7f1734160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 -- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17340cc6a0 msgr2=0x7f1734160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17340cc6a0 0x7f1734160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 -- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734160750 msgr2=0x7f17341f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734160750 0x7f17341f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.981+0000 7f173915f640 1 -- 172.21.15.105:0/3140459333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17340c8e70 con 0x7f1720003a30 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f173915f640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1730036040 tx=0x7f1730033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730038040 con 0x7f1720003a30 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f1730004020 con 0x7f1720003a30 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173415d620 con 0x7f1720003a30 2024-09-21T07:28:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730004450 con 0x7f1720003a30 2024-09-21T07:28:01.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.982+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173415dbe0 con 0x7f1720003a30 2024-09-21T07:28:01.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.983+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17340d0010 con 0x7f1720003a30 2024-09-21T07:28:01.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.985+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 14) ==== 101441+0+0 (secure 0 0 0) 0x7f1730045020 con 0x7f1720003a30 2024-09-21T07:28:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.985+0000 7f1729ffb640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 0x7f171807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:28:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.985+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5937+0+0 (secure 0 0 0) 0x7f17300bb7a0 con 0x7f1720003a30 2024-09-21T07:28:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.986+0000 7f173895e640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 0x7f171807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:28:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.986+0000 7f173895e640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 0x7f171807b0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f173406e8e0 tx=0x7f1724006720 comp rx=0 tx=0).ready entity=mgr.5364 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:28:01.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:01.990+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17300834a0 con 0x7f1720003a30 2024-09-21T07:28:02.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.213+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1734161fc0 con 0x7f1720003a30 2024-09-21T07:28:02.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.214+0000 7f1729ffb640 1 -- 172.21.15.105:0/3140459333 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3917 (secure 0 0 0) 0x7f1730087ed0 con 0x7f1720003a30 2024-09-21T07:28:02.216 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-09-21T07:28:02.216 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":20,"btime":"2024-09-21T07:27:30:188822+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4700,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1462496078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1462496078},{"type":"v1","addr":"172.21.15.105:6835","nonce":1462496078}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/1920414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1920414291},{"type":"v1","addr":"172.21.15.112:6836","nonce":1920414291}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4706,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/3411603175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":3411603175},{"type":"v1","addr":"172.21.15.112:6837","nonce":3411603175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5415,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/95339601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":95339601},{"type":"v1","addr":"172.21.15.105:6833","nonce":95339601}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T07:28:02.216 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 20 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.216+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 msgr2=0x7f171807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.216+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 0x7f171807b0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f173406e8e0 tx=0x7f1724006720 comp rx=0 tx=0).stop 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.216+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 msgr2=0x7f17340cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.216+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1730036040 tx=0x7f1730033670 comp rx=0 tx=0).stop 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 shutdown_connections 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1734160750 0x7f17341f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:02.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:6800/440752644,v1:172.21.15.112:6801/440752644] conn(0x7f1718078bc0 0x7f171807b0d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:02.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17340cc6a0 0x7f1734160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:02.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 --2- 172.21.15.105:0/3140459333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1720003a30 0x7f17340cdbf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:28:02.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 >> 172.21.15.105:0/3140459333 conn(0x7f173408bfb0 msgr2=0x7f173406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:28:02.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 shutdown_connections 2024-09-21T07:28:02.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-09-21T07:28:02.217+0000 7f173b3c3640 1 -- 172.21.15.105:0/3140459333 wait complete. 2024-09-21T07:28:02.230 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T07:28:02.230 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T07:28:02.231 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:28:02.231 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.243 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:28:02.243 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:28:02.243 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.292 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:28:02.292 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:28:02.292 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.305 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:28:02.305 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:28:02.305 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.357 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:28:02.357 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T07:28:02.357 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T07:28:02.357 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.391 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T07:28:02.391 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T07:28:02.391 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.459 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T07:28:02.459 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T07:28:02.460 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.527 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T07:28:02.528 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T07:28:02.528 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.600 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T07:28:02.600 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T07:28:02.600 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.635 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T07:28:02.635 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T07:28:02.635 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.703 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T07:28:02.704 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T07:28:02.704 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.769 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T07:28:02.769 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T07:28:02.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.836 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T07:28:02.838 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T07:28:02.839 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T07:28:02.839 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.871 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T07:28:02.871 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T07:28:02.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.910 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T07:28:02.910 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T07:28:02.910 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T07:28:02.910 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.922 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T07:28:02.922 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T07:28:02.922 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:02.962 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T07:28:02.963 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T07:28:02.963 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:28:03.015 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T07:28:03.015 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T07:28:03.016 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-21T07:28:03.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:28:03.062 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T07:28:03.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:28:03.221 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T07:28:03.316 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:28:03.317 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T07:28:03.407 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:28:03.408 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T07:28:03.466 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:28:03.466 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T07:28:03.625 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:28:03.625 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T07:28:03.735 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:28:03.736 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T07:28:03.793 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:28:03.794 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T07:28:03.851 INFO:tasks.ceph:Archiving mon data... 2024-09-21T07:28:03.853 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/data/mon.a.tgz 2024-09-21T07:28:03.853 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-09-21T07:28:03.869 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.GhiMgghLOn 2024-09-21T07:28:03.869 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.GhiMgghLOn 2024-09-21T07:28:04.066 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.GhiMgghLOn 2024-09-21T07:28:04.150 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.GhiMgghLOn is 725KB 2024-09-21T07:28:04.251 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.GhiMgghLOn 2024-09-21T07:28:04.267 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/data/mon.b.tgz 2024-09-21T07:28:04.270 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T07:28:04.287 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.DcIAYvCy1F 2024-09-21T07:28:04.287 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.DcIAYvCy1F 2024-09-21T07:28:04.532 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.DcIAYvCy1F 2024-09-21T07:28:04.615 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.DcIAYvCy1F is 1MB 2024-09-21T07:28:04.692 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.DcIAYvCy1F 2024-09-21T07:28:04.707 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/data/mon.c.tgz 2024-09-21T07:28:04.707 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-09-21T07:28:04.761 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.D6jRESh3hg 2024-09-21T07:28:04.762 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.D6jRESh3hg 2024-09-21T07:28:04.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.D6jRESh3hg 2024-09-21T07:28:05.073 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.D6jRESh3hg is 1MB 2024-09-21T07:28:05.175 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.D6jRESh3hg 2024-09-21T07:28:05.191 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T07:28:05.191 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:28:05.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:28:05.276 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:28:05.312 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:28:05.312 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T07:28:05.314 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi105/crash 2024-09-21T07:28:05.315 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:28:05.343 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi112/crash 2024-09-21T07:28:05.344 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:28:05.377 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T07:28:05.378 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:28:05.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:28:05.714 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:28:05.740 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:28:05.765 INFO:tasks.ceph:Compressing logs... 2024-09-21T07:28:05.766 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:28:05.783 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42430.log 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36787.log 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36547.log 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.1.42430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42430.log.gz 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36787.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36547.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36787.log.gz 2024-09-21T07:28:05.859 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42827.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36547.log.gz 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.3.42827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42827.log.gz 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36587.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36747.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.36747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36707.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36747.log.gz 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36707.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T07:28:05.860 INFO:teuthology.orchestra.run.smithi112.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36707.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36667.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36627.log: /var/log/ceph/ceph-client.admin.36667.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36667.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36827.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mgr.x.log: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T07:28:05.861 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36827.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36827.log.gz 2024-09-21T07:28:05.879 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T07:28:05.910 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T07:28:05.910 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46168.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50050.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36858.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46168.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50441.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50050.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36858.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49387.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49387.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49387.log.gz 2024-09-21T07:28:05.911 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47437.log: 87.5%/var/log/ceph/ceph-client.admin.50109.log: -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36960.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45029.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48434.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45029.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48890.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45028.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48434.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43320.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48890.log.gz 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36995.log 2024-09-21T07:28:05.912 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45028.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43320.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44806.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36995.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49809.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44806.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.49809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45361.log 2024-09-21T07:28:05.913 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49809.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51722.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45361.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45694.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45602.log 2024-09-21T07:28:05.914 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48182.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50672.log: /var/log/ceph/ceph-client.admin.45602.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43020.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45602.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46695.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48182.log: /var/log/ceph/ceph-client.admin.43020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48182.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43020.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.0.46695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46695.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45700.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44041.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-09-21T07:28:05.915 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45700.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43192.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36506.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36776.log: /var/log/ceph/ceph-client.admin.43192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45024.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43192.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43446.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36506.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45702.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45024.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.44041.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43064.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.45024.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43446.log.gz 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47616.log 2024-09-21T07:28:05.916 INFO:teuthology.orchestra.run.smithi105.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45702.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43236.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43064.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43064.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47616.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46210.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43236.log: /var/log/ceph/ceph-client.admin.49513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43236.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42978.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49261.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46210.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51932.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.42978.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46126.log 2024-09-21T07:28:05.917 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42978.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49261.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47574.log 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51932.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46126.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37000.log 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47574.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42918.log 2024-09-21T07:28:05.918 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37000.log: /var/log/ceph/ceph-client.admin.49983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.37000.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.42918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42918.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45364.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49429.log: /var/log/ceph/ceph-client.admin.49303.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30350.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48224.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45364.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.tmp-client.admin.30350.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30350.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44470.log 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48224.log: /var/log/ceph/ceph-client.admin.43596.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48224.log.gz 2024-09-21T07:28:05.919 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48014.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50798.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44470.log: /var/log/ceph/ceph-client.admin.48014.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44470.log.gz 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48014.log.gz 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45898.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50798.log.gz 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50630.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.45898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48806.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45898.log.gz 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52058.log 2024-09-21T07:28:05.920 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50630.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42836.log 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48806.log: /var/log/ceph/ceph-client.admin.52058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49931.log 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48806.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52058.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45366.log 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.42836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5%/var/log/ceph/ceph-client.admin.49931.log: -- replaced with /var/log/ceph/ceph-client.admin.42836.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45366.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49931.log.gz 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45356.log 2024-09-21T07:28:05.921 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45366.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51806.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45356.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz -5 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43150.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45356.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36999.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51806.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48721.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43150.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36999.log: /var/log/ceph/ceph-client.admin.48721.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36999.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48721.log.gz 2024-09-21T07:28:05.922 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48476.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45695.log: /var/log/ceph/ceph-client.admin.47769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51386.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48476.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51386.log: /var/log/ceph/ceph-client.admin.50204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37002.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51386.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50577.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37002.log: /var/log/ceph/ceph-client.admin.50577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37002.log.gz 2024-09-21T07:28:05.923 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44309.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50577.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51344.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36589.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47937.log: /var/log/ceph/ceph-client.admin.51344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51344.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.36589.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45982.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-09-21T07:28:05.924 INFO:teuthology.orchestra.run.smithi105.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44428.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.44722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50985.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44428.log.gz 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48266.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51218.log: /var/log/ceph/ceph-client.admin.50985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51554.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50985.log.gz 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2024-09-21T07:28:05.925 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48266.log.gz 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45027.log 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51554.log.gz 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43278.log 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36730.log: /var/log/ceph/ceph-client.admin.45027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37376.log 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43278.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45027.log.gz 2024-09-21T07:28:05.926 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51764.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43278.log.gz 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44848.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37376.log.gz 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51764.log.gz 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44848.log.gz 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45363.log: /var/log/ceph/ceph-client.admin.50714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36959.log 2024-09-21T07:28:05.927 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51596.log 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36959.log: /var/log/ceph/ceph-client.admin.48637.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36959.log.gz 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36562.log 2024-09-21T07:28:05.928 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36688.log 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51596.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36562.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51596.log.gz 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.36562.log.gz 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.36688.log.gz 2024-09-21T07:28:05.929 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48308.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51680.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36976.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48308.log.gz 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51680.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47531.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36976.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51680.log.gz 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36976.log.gz 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49113.log 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47531.log: /var/log/ceph/ceph-client.admin.49155.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47531.log.gz 2024-09-21T07:28:05.930 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47395.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49062.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49113.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47395.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45701.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49062.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/ceph-client.admin.47487.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47847.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45701.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36477.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45701.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47847.log.gz 2024-09-21T07:28:05.931 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51302.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr: 74.4% -- replaced with /var/log/ceph/ceph-client.admin.36477.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49621.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44638.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51302.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48848.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49621.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43362.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48848.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48848.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45600.log/var/log/ceph/ceph-client.admin.43362.log: 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51512.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43362.log.gz 2024-09-21T07:28:05.932 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48932.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3%/var/log/ceph/ceph-client.admin.51428.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.45600.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36917.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48932.log: /var/log/ceph/ceph-client.admin.36917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43108.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.36917.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48932.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48140.log 2024-09-21T07:28:05.933 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43108.log.gz 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48140.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48140.log.gz 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50943.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51890.log 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42659.log 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50943.log.gz 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.42659.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51890.log.gz 2024-09-21T07:28:05.934 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42659.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52118.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37004.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49197.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52118.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph.log: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48098.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49888.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49197.log: /var/log/ceph/ceph-client.admin.48098.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49197.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48098.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50322.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49888.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49888.log.gz 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51260.log 2024-09-21T07:28:05.935 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45030.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50322.log: /var/log/ceph/ceph-client.admin.51260.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50322.log.gz 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51260.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51470.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45030.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45030.log.gz 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48974.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51470.log.gz/var/log/ceph/ceph-client.admin.48974.log: 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52201.log 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48974.log.gz 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.43488.log: gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47352.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50162.log 2024-09-21T07:28:05.936 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52201.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48350.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47352.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43530.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50162.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48350.log: /var/log/ceph/ceph-client.admin.50840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47658.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48350.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz/var/log/ceph/ceph-client.admin.43530.log: 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52159.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47658.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43530.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47658.log.gz 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log 2024-09-21T07:28:05.937 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52159.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.47268.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.2.47134.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45360.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.2.47134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.47134.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52016.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7%/var/log/ceph/ceph-client.admin.49345.log: -- replaced with /var/log/ceph/ceph-client.admin.45360.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37577.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36996.log 2024-09-21T07:28:05.938 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.52016.log: /var/log/ceph/ceph-client.admin.37577.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52016.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48764.log 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37577.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51027.log 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48764.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36996.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46375.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48764.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45354.log 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51027.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50535.log 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46375.log.gz 2024-09-21T07:28:05.939 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45354.log.gz 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46084.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50535.log.gz 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36450.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44764.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.46084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48679.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46084.log.gz 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48392.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44764.log.gz 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42740.log 2024-09-21T07:28:05.940 INFO:teuthology.orchestra.run.smithi105.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48679.log.gz 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45020.log 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48392.log.gz 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.42740.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42740.log.gz 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.45020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45020.log.gz 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz/var/log/ceph/ceph-mds.c.log: 2024-09-21T07:28:05.941 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48056.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.49471.log: /var/log/ceph/ceph-client.admin.45601.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36393.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.48056.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48056.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51974.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.36393.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36393.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51974.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51974.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.44251.log 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50901.log: gzip -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-09-21T07:28:05.942 INFO:teuthology.orchestra.run.smithi105.stderr: -- /var/log/ceph/ceph-client.admin.51848.log 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.44251.log: gzip -5 --verbose -- 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr: /var/log/ceph/ceph-client.admin.37666.log 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.51848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50756.log 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51848.log.gz 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.37666.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37666.log.gz 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/ceph-client.admin.50756.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50756.log.gz 2024-09-21T07:28:05.943 INFO:teuthology.orchestra.run.smithi112.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T07:28:05.944 INFO:teuthology.orchestra.run.smithi112.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T07:28:05.960 INFO:teuthology.orchestra.run.smithi112.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T07:28:06.085 INFO:teuthology.orchestra.run.smithi105.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T07:28:06.459 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T07:28:06.486 INFO:teuthology.orchestra.run.smithi112.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T07:28:06.496 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T07:28:06.504 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T07:28:06.541 INFO:teuthology.orchestra.run.smithi112.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T07:28:06.562 INFO:teuthology.orchestra.run.smithi105.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T07:28:06.568 INFO:teuthology.orchestra.run.smithi112.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T07:28:06.570 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-09-21T07:28:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m0.744s 2024-09-21T07:28:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m2.953s 2024-09-21T07:28:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.130s 2024-09-21T07:28:06.627 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T07:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-09-21T07:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m0.834s 2024-09-21T07:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m3.086s 2024-09-21T07:28:06.629 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.213s 2024-09-21T07:28:06.630 INFO:tasks.ceph:Archiving logs... 2024-09-21T07:28:06.630 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi105/log 2024-09-21T07:28:06.631 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:28:07.237 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi112/log 2024-09-21T07:28:07.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:28:07.659 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T07:28:07.667 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T07:28:07.668 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:28:07.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:28:07.751 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T07:28:07.761 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T07:28:07.761 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:28:07.763 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:28:07.777 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:28:07.778 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv01.front.sepia.ceph.com 3 6 377 8 +38us[ +39us] +/- 54ms 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 8 -172us[ -172us] +/- 64ms 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 8 +148us[ +150us] +/- 68ms 2024-09-21T07:28:07.781 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:28:07.782 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:28:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-09-21T07:28:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 7 +46us[ +48us] +/- 54ms 2024-09-21T07:28:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -172us[ -172us] +/- 64ms 2024-09-21T07:28:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +146us[ +146us] +/- 68ms 2024-09-21T07:28:07.783 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:28:07.783 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T07:28:07.792 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T07:28:07.792 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T07:28:07.802 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:28:07.893 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:28:07.957 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:28:07.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:07.985 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:28:08.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:08.013 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T07:28:08.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T07:28:08.032 INFO:teuthology.task.internal:Duration was 497.088861 seconds 2024-09-21T07:28:08.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T07:28:08.046 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T07:28:08.046 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:28:08.049 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:28:08.086 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:28:08.092 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:28:08.394 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T07:28:08.394 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-09-21T07:28:08.395 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:28:08.442 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-09-21T07:28:08.442 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:28:08.463 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T07:28:08.464 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:28:08.485 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:28:08.529 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T07:28:08.530 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:28:08.552 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:28:08.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T07:28:08.680 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T07:28:08.680 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:28:08.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:28:08.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T07:28:08.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:08.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-09-21T07:28:08.793 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-09-21T07:28:08.806 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:08.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:08.835 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:28:08.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:28:08.859 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T07:28:08.869 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T07:28:08.870 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi105 2024-09-21T07:28:08.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:28:08.909 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913597/remote/smithi112 2024-09-21T07:28:08.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:28:08.942 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T07:28:08.942 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:28:08.946 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:28:08.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T07:28:09.003 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T07:28:09.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T07:28:09.013 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T07:28:09.014 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:28:09.016 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:28:09.029 INFO:teuthology.orchestra.run.smithi105.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:28 /home/ubuntu/cephtest 2024-09-21T07:28:09.046 INFO:teuthology.orchestra.run.smithi112.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:28 /home/ubuntu/cephtest 2024-09-21T07:28:09.047 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T07:28:09.057 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T07:28:09.111 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/snap_schedule_snapdir} duration: 497.0888614654541 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T07:28:09.111 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:28:09.188 INFO:teuthology.run:pass